{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "a06626b1-8b12-4482-87be-149128f7fb52",
   "metadata": {},
   "source": [
    "# 文心大模型 DPO 最佳实践"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "426c7c9d",
   "metadata": {},
   "source": [
    "# 1 DPO 简介  \n",
    "### 1.1 什么是 DPO\n",
    "DPO（Direct Preference Optimization，直接偏好优化）是一种通过人类偏好数据优化语言模型的方法，旨在让模型输出更符合人类期望，绕过了传统RLHF（Reinforcement Learning from Human Feedback）中的奖励模型训练和强化学习步骤。训练数据需要同时带有正例（期望的输出）、负例（不期望的输出）。\n",
    "\n",
    "### 1.2 DPO 适用范围\n",
    "- ✅ 适用\n",
    "  - 行为对齐，对齐人类偏好、价值观。例如让回复更友好、礼貌或安全；\n",
    "  - 对于答案不唯一的开放性问题（如内容创作、摘要、对话），让模型学会更细致、更受欢迎的表达方式；\n",
    "  - 强化特定约束或格式。例如期望模型的回复更简短。\n",
    "\n",
    "- ❌ 不适用\n",
    "  - 给模型添加新的内化知识；\n",
    "  - 强迫模型学会不具备的能力；\n",
    "  - 完全纠正预训练模型中的固有偏差；\n",
    "  - 单一正确答案的任务（如严格的数学计算）。\n",
    "\n",
    "\n",
    "### 1.3 DPO 的一般步骤\n",
    "1. 构造评估数据、评估标准，测试模型基础能力，分析其短板、问题。\n",
    "2. 进行充分的 prompt 工程。\n",
    "3. 判断是否适用 DPO；准备训练数据。\n",
    "4. 训练。结合任务特点、数据情况，设计训练参数。一般关注以下核心参数：\n",
    "    - 批处理大小\n",
    "    - 训练轮数、最大训练步数\n",
    "    - 学习率类型、学习率初始值\n",
    "    - 预热步数\n",
    "5. 效果评估。采用相同的评估数据、评估标准，评估训练后的模型表现。若效果不达标，考虑继续优化训练数据、调整训练参数。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bd37505",
   "metadata": {},
   "source": [
    "# 2 最佳实践任务简介\n",
    "### 2.1 任务描述\n",
    "我们观察到一些场景下模型的回复较为冗长，我们希望让模型的回复更短小、精炼。\n",
    "\n",
    "### 2.2 数据来源\n",
    "我们使用开源社区内较为流行的 llamafactory 提供的 DPO demo 数据集（[链接](https://huggingface.co/datasets/llamafactory/DPO-En-Zh-20k/tree/main)）。该数据集有中、英文数据各1千条，采用 ShareGPT 格式，具有以下字段：\n",
    "- `conversations`：对话上下文，[{\"from\": \"\", \"value\": \"\"}] 格式，其中 `from` 字段包括\n",
    "  - `system`：system prompt\n",
    "  - `human`：用户输入\n",
    "  - `gpt`：模型输入（这里用 gpt 指代了大模对话助手）\n",
    "- `chosen`：正例（期望的输出）\n",
    "- `rejected`：负例（不期望的输出）\n",
    "> 真实场景下，我们一般基于模型当前输出，构造更优数据，形成训练样本；本教程目的在于示范 DPO 流程，因此直接下载了开源数据集。\n",
    "\n",
    "### 2.3 评估标准\n",
    "我们关注输出的长度，因此可以用字符串长度的平均值、中位数评价模型表现。\n",
    "\n",
    "### 2.4 实验环境\n",
    "本教程采用以下环境：\n",
    "- 1张 80GB A800 GPU\n",
    "- CUDA 版本：12.3\n",
    "- CUDA 驱动：525.125.06\n",
    "- nvcc：12.3\n",
    "- gcc：12.2\n",
    "- Python 版本：3.10.12\n",
    "\n",
    "### 2.5 依赖项\n",
    "- **ERNIEKit**：文心大模型工具链，包含文心4.5系列模型训练-压缩-推理的全流程使用，基于飞桨框架v3.1可在多款主流国产芯片上进行训练。\n",
    "- **ERNIEKit WebUI**：可视化界面，支持训练、对话交互、性能评估、模型导出等功能。[文档](../../docs/cli_webui_usage.md)\n",
    "- **[可选]visualdl**：可视化 loss 等信息的工具，ERNIEKit 已包含。\n",
    "- **ERNIEKit 推理脚本**。\n",
    "- 本教程中的 **Python 依赖**。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "91ab45af",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 本教程中的 Python 依赖\n",
    "import json\n",
    "import random\n",
    "from collections import defaultdict\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "\n",
    "RANDOM_SEED = 2025"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a6d8592",
   "metadata": {},
   "source": [
    "# 3 数据预处理"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0612a29c",
   "metadata": {},
   "source": [
    "### 3.1 准备原始数据\n",
    "首先下载开源数据集，保存到 `cookbook/data/llamafactory_dpo_zh_10k.json`  \n",
    "\n",
    "![dpo_download_data](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_download_dataset_en.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9752ae46",
   "metadata": {},
   "source": [
    "为了便于处理，这里将 ShareGPT 格式转为 ERNIE DPO 格式。我们定义了若干数据预处理的函数。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ea18004a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def conversations_to_ernie_data(conversations):\n",
    "    system = \"\"\n",
    "    src = []\n",
    "    tgt = []\n",
    "    for conversation in conversations:\n",
    "        if conversation.get(\"from\", \"\") == \"system\":\n",
    "            system = conversation.get(\"value\", \"\")\n",
    "        elif conversation.get(\"from\", \"\") == \"human\":\n",
    "            src.append(conversation.get(\"value\", \"\"))\n",
    "        elif conversation.get(\"from\", \"\") == \"gpt\":\n",
    "            tgt.append(conversation.get(\"value\", \"\"))\n",
    "    return system, src, tgt\n",
    "\n",
    "\n",
    "def is_valid_ernie_data(data):\n",
    "    if len(data[\"src\"]) == 0:\n",
    "        return False\n",
    "    if len(data[\"response\"][0]) == 0 or len(data[\"response\"][1]) == 0:\n",
    "        return False\n",
    "    return True"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6be94810",
   "metadata": {},
   "source": [
    "调用以上函数处理数据。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "94660786",
   "metadata": {},
   "outputs": [],
   "source": [
    "raw_data_path = \"../data/llamafactory_dpo_zh_10k.json\"\n",
    "ernie_data_path = raw_data_path.replace(\".json\", \"_ernie_format.jsonl\")\n",
    "\n",
    "with open(raw_data_path, \"r\") as fin:\n",
    "    with open(ernie_data_path, \"w\") as fout:\n",
    "        data_list = json.load(fin)\n",
    "        for data in data_list:\n",
    "            system, src, tgt = conversations_to_ernie_data(data.get(\"conversations\", []))\n",
    "            chosen = data.get(\"chosen\", {}).get(\"value\", \"\")\n",
    "            rejected = data.get(\"rejected\", {}).get(\"value\", \"\")\n",
    "            ernie_data = {\n",
    "                \"system\": system,\n",
    "                \"src\": src,\n",
    "                \"tgt\": tgt,\n",
    "                \"response\": [[chosen], [rejected]],\n",
    "                \"sort\": [1, 0],\n",
    "            }  # sort 值小的是 rejected，大的是 chosen\n",
    "            if is_valid_ernie_data(ernie_data):\n",
    "                fout.write(json.dumps(ernie_data, ensure_ascii=False) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03201de0",
   "metadata": {},
   "source": [
    "### 3.2 分析原始数据\n",
    "观察数据集中 chosen、rejected 的长度分布，然后针对性构造测试集、训练集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "62eedc9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "chosen_len_cnt = defaultdict(int)\n",
    "rejected_len_cnt = defaultdict(int)\n",
    "\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    for line in fin:\n",
    "        data = json.loads(line)\n",
    "        chosen_len = len(data[\"response\"][0][0])\n",
    "        rejected_len = len(data[\"response\"][1][0])\n",
    "        chosen_len_cnt[chosen_len] = chosen_len_cnt.get(chosen_len, 0) + 1\n",
    "        rejected_len_cnt[rejected_len] = rejected_len_cnt.get(rejected_len, 0) + 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f0719d6",
   "metadata": {},
   "source": [
    "做简单可视化分析"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "944b5348",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABdEAAAJOCAYAAABYwk4SAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8ekN5oAAAACXBIWXMAAA9hAAAPYQGoP6dpAACopElEQVR4nOzdfXzN9f/H8efZpclVczHm+lqYbRFKClGUSnPtm4svhZBvuYjlolwlpJDlouQiJdeJilIu8i3UMEWEsBljw0g2Zzs7vz/23ee3YzucMzvbweN+u3XjfN7vz/m8P+/3cXp9Xuf9eX9MVqvVKgAAAAAAAAAAkIVHfjcAAAAAAAAAAAB3RRIdAAAAAAAAAAA7SKIDAAAAAAAAAGAHSXQAAAAAAAAAAOwgiQ4AAAAAAAAAgB0k0QEAAAAAAAAAsIMkOgAAAAAAAAAAdpBEBwAAAAAAAADADpLoAAAAAAAAAADY4ZXfDQBuJyNHjtTatWtttplMJvn5+alMmTJ65JFH9O9//1sBAQF29/v+++9Vrly5HB0/LS1NR44cUc2aNR2qf+rUKT322GOSpOeee05vv/22JKl79+7avXu3JOnw4cM5aosjYmJidO+996pQoUKSpF27dqlHjx6SpEGDBunll1922bFdYffu3XrnnXd0+PBheXp6qmbNmvrss89kMpns7pOcnKxly5Zp06ZNOnbsmJKSklSiRAnVr19fPXr0UHBwsE39vBobd9SiRQvFxsZm2e7p6amCBQuqQoUKat26tXr16iUfH598aOHdI+M7pmHDhvrkk0/yuTXOu3Llii5evKjy5csb2273cwKAuxGxt3Pu5tg7cx9n5uHhIT8/PwUGBqpFixbq27ev0T/OyohVy5Ytqx9++CFH7+Gs68c0t7Rq1UrR0dE3PZfMn+nreXt7q3DhwqpWrZq6deumNm3a5GobYet2//cs3XnfUbj7MBMduEVWq1VXr17VsWPHtHDhQj333HM6dOhQrh/nxx9/1LPPPquFCxfm+nvntsTERE2ePFlt2rRRYmJifjcnVyQlJemll15SVFSUkpOT9c8//+jvv/++YQL9+PHjeuaZZ/T2229r7969unz5slJSUnTmzBlt2LBBXbp00YIFC/LwLG5PFotFf//9tw4cOKDp06dryJAh+d0kuKnU1FR9+umnatWqlX755Zf8bg4AwAWIvbMi9rYvLS1N//zzj44cOaJ58+apd+/eSklJcVGrc8/tMKYpKSm6cOGCdu/erVdeeUVLlizJ7ybBTd0On2fAEcxEB3Jo+fLlKl26tFJSUnT27FmtW7dOK1as0Pnz5/Xyyy9rw4YN8vX1lSSFh4frlVdekSSVLFnS6WPFxsbqhRdekCTVqVPH4f3KlCmjbdu2SZL8/PycPm5OTZkyRWvWrMmyPTQ01GhPbs+mcLWjR4/qypUrkqSHH35YY8eOVVpamt36ly9f1gsvvKBTp07Jw8NDffr0Udu2beXh4aGtW7dq9uzZunbtmqZNm6a6deuqUaNGeXUqbq906dJavny58To1NVXHjh3T66+/roSEBH333Xfas2eP7r///nxsJdzR+vXrNX78+PxuBgDABYi97SP2tpVxzlL6ZIzTp09rzJgxOn78uKKiorRx40Y9/fTTTrdp+fLlslgs8vT0dHpfZ9kb0/zUunVrhYeHS0r/MSs1NVXbt2/XxIkTlZaWplmzZqljx455+tnH7cEdP89ATpBEB3KoRIkSKl26tCSpfPnyatCggTw8PPT5558rOjpa69atU6dOnSRJRYsWVdGiRXN8LKvVmqP9PD09jTbmJXvt9fHxyZf25IakpCTj78HBwapYseIN6y9YsECnTp2SJI0YMUK9evUyymrUqKF7771Xo0ePltVq1Zo1a0iiZ5Ld57ZcuXLq2bOnpk+fLknav38/SXRkkdPvSgCA+yP2to/Y29b151y2bFkNHjxYr776qiQpKioqR0n0nPwgk1PuGNP4+fll6dt//etf2rJli3788Uf9/fffOn78uGrXrp1PLYS7csfPM5ATLOcC5KLu3bsbf8+8ttzIkSNVs2ZN1axZ00isStLevXvVv39/Pfjgg6pdu7buv/9+dezYUatXrzbqrFmzxmYdurVr16pmzZrGL7ktWrRQzZo1NXjwYH300Udq1KiRQkJCNGPGDJ06dco47siRI7Ntc2xsrF5++WWFhoaqQYMGGjRokI4fP57lvDLeJ7Ndu3YZ299//32jPZnXrnzsscfUokULu/UzXLlyRTNnztRTTz2l4OBgNWjQQD179tR3332Xpc2Zz/mvv/7SgAEDVL9+fYWGhqp///5Z2m9PWlqaVqxYoS5duqhBgwYKCQlRu3bt9PHHH8tsNtucf+axjYiIsBmD7GT0QdGiRfWvf/0rS/mzzz6rt99+W999952mTJmS7XtcuHBBr7/+uho1aqTQ0FD16dMn29uVz507p7feekuPP/64goKC1KhRI/Xv3z/bdSGPHTumV155RQ8//LDq1KmjkJAQPfvss/roo4+yzO5JS0vTkiVL9PTTTysoKEgNGzZU//79tX//fpt6mcd1y5Yt+uKLL/TMM88oKChIzZo103vvvZcrt816ef3/774FChSwKdu+fbu6d++u0NBQhYaGqlOnTvryyy9zfP6Z/+0sWrRI69evN87pscce09y5c2WxWLK8/2+//aahQ4eqadOmqlu3rh599FGNGjVKMTExNvVy0mdbt25Vz5491bBhQ9WuXVsPPPCAnn/+eX3//fdZ2nH58mVNnjxZLVq0UN26ddW0aVONGTNGZ8+edayznXTt2jXNnj1bTzzxhOrWrauHHnpIQ4YM0V9//WVTb82aNcZ5Hzp0SIsWLTL2efzxx7O9Zf7ChQsaO3asmjRpouDgYP3rX//Svn37jO/VjO+XkSNHGrOjpPRZiNd/52Y4dOiQevfurZCQEDVu3FijRo3ShQsXbOo48v0MAMhfxN7E3o7KPHv8+pnSUVFR6tu3rxo0aKB69erp2Wef1SeffJIlNs7oh4z+zeBM3HX58mVNmzZNTzzxhIKCgvTQQw+pZ8+eNrPnbzSmkuNxlySdPXtWI0eOVOPGjRUcHKxevXrl+vJH9mJ0R68lJMfj3Myx5P79+/X++++refPmCgoKUlhYmL799tts27hx40b16tVLjRo1UlBQkJ588knNnDnTuNshw/vvv2+8/8WLFzVz5kw1b95cdevW1dNPP60vvvjCpn5aWpoWLVqk5557TqGhoapTp46aNGmil19+WUeOHMnSjoxrkcaNGysoKEitW7dWRESErl27dtN+zomzZ89q1KhRevjhh1W3bl21aNFCb7/9ti5fvmxTL+M7s1GjRvr77781YcIEPfzwwwoKClKHDh20ffv2LO996NAh9evXT/fff7/q16+vIUOG6Ny5c8a/k4zvwJt9njP75ptv9OyzzyooKEgtWrTQBx98kOWa64svvlCXLl1Uv3591a5dW40aNVKfPn0UGRl5q90F3BQz0YFcVLVqVRUoUEDJycn6448/blh3//796t69u02i7J9//tH+/fu1f/9+Xb58Wf/+978dPvbPP/+sTZs2Ga+vf2ClPV26dNG5c+eM199995127dqllStXqlKlSg4f/1acPXtW3bp1s7nISU5O1s6dO7Vz50717NlTr7/+epb9jh8/rk6dOunvv/82tm3ZskVHjhzRpk2bbAK661ksFvXv3z9LQPDHH3/ojz/+0HfffacFCxaoYMGCTp/PmTNnjKC5bt268vb2zlLHx8dHzz333A3fp1u3bjYXJTt27NDvv/+uH374Qffcc4+k9OClV69eunjxolHPbDZry5Yt2rp1q0aOHGnMgo+NjVXnzp1t+is1NVWHDh3SoUOHFBcXp9GjRxtlw4YN01dffZXlfXfs2KHZs2erWbNmWdr80Ucf6ddff7Xpi7lz58rDw0P/+c9/bni+2bFarUpKStKff/5pPAzS09NTDzzwgFHns88+0/jx421mOERFRSkqKkpHjx411lB39vwzrFmzxuYhYKdOndJ7772nI0eOGDPjpfSAbtSoUUpNTTW2xcXFadWqVdq4caPmzZunBg0aZHl/R/rsu+++08svv2xzjpcvX9Yvv/yiX3/9VTNmzFDr1q0lSZcuXVKXLl1sLqTOnTunFStWaMuWLVq+fLnKli1rt8+dZTab1bt3b5tzOH/+vL766itt3bpVS5YsUd26dbPsN2HCBJt9Tp48qbfffluFCxdWhw4dJEl///23unbtqhMnThj1fv31V/Xo0UOVK1fOUXtPnjyprl276urVq5LSZ7qtWrVKcXFxxjMKcvv7GQDgGsTeOXOnxd43kpycrJMnT2rOnDnGtsx3gH7//ff6z3/+Y/O5OHTokCZOnKh9+/bZxHrZcSbuunjxorp06WIT15w/f17nz5/Xzp07NXHiRHXs2PGGx3Mm7rpw4YI6d+6sM2fOGHV//vlndevW7YbHcERaWpquXLmi7du3a8eOHZLSlzOqUKGCUcfRawln4tzMRo8ebROjHzhwQC+//LLGjRunLl26GNvHjh1rs1SklJ7M/uCDD7Rx40Z98sknKlGiRJb3HzRokE0///nnnxoxYoQCAgL04IMPSpLefvttLV682Ga/hIQEffvtt8a/64y7KPbv369evXrpn3/+MeoeP35cs2bN0s8//6yFCxdme92YUzExMeratavi4+ONbbGxsVq4cKG2b9+u5cuXq3Dhwjb7pKSkqHv37jbfp7/99pv69++vr776yoi/f/vtN3Xv3t3mjpGvvvpKv/32mxFjO+urr76yufaNjY3VzJkz5enpqX79+kmSFi9erLfeestmv8TERO3YsUO7d+/W4sWLuVsZLsVMdCAXmUwm439Ely5dumHddevWKSUlRQULFtRHH32kzZs367PPPlP16tXl5eWljRs3Ki0tTW3atLH5n37r1q21bdu2LE8/v3z5sp544gl98803eu+999S0aVOH2lywYEF9/PHHWrdunZHUvXz5st555x1nTt2wfPlymyBn+fLlWYKW673++utGEN+jRw+tW7dOixYtMtagXLx4sU0AluHPP/9U3bp1tWrVKq1cuVIBAQGS0pOcN3uo4IIFC4wg/sEHH9Tnn3+uNWvWqG3btpKkPXv26O2335YkzZw5UzNmzDD27dWrV7ZjkCFzoHLvvffesB03UrBgQX322Wdav369cWGWmJhozBBKS0vTkCFDdPHiRXl6euo///mPvvrqK82ZM0fly5eX1WrV22+/rT179kiSNm3aZFz0zJgxQ5s3b9bq1avVsGFDeXh46McffzRmY3zzzTdGn7dr107r16/X8uXLFRoaqpSUFL3++us2M4YyREZGasiQIdq0aZNGjRplbHdm5lBsbKwxA6RWrVoKDQ1V586ddfr0aZlMJg0aNEhVq1aVlH4R+NZbb8lqtSooKEifffaZvvrqK+MCZP78+Tpw4IDT55/Z4cOHFRYWpnXr1umjjz5SuXLlJEkbNmwwLhpiYmI0duxYpaam6t5779W0adP09ddf64033lDBggV15coVvfzyy9m+vyN9tnr1almtVpUpU0affvqpNm/erI8//lilSpWSl5eXvv76a6PujBkz9Ndff8lkMmnkyJHauHGj5syZo5IlSyo+Pl4TJ050eCwcsWTJEuMC44UXXtDXX3+txYsXq0qVKvrnn39szimz/fv3a8KECfr222/14osvZnveH374oXGh2aRJE61atUqff/65goKCssyiCg8PzzITfdu2bSpTpoxNvbNnz+qxxx7Thg0btGTJEhUrVkxS+o9UGReZjn4/AwDyF7E3sXd2MuLImjVrKjg4WM8884yRFOzYsaMefvhhSek/pI8ePVopKSmqUKGCFixYoG+++UYDBgyQlB7rZTczPzNn4q733nvPiGv+9a9/af369Vq8eLGRZJ8yZYquXLlywzF1Ju6aO3euEds0bdpUq1ev1meffabKlSvbJHIdlXFXRs2aNXXffffpgQce0NChQ41/VxMmTDB+SHHmWsKZODezY8eOacSIEfr66681ceJEYxb8lClTjO+Dr776yui7++67T4sWLdKXX35pTDL666+/NGLEiGzf/8iRI5oxY4Y2bdqkZ5991tieOVZduXKlpPR1+7/44gtt3rxZEydOlKenpywWizZv3iwpfWLQqFGj9M8//+jee+/V7NmztXHjRo0ZM0YeHh765Zdf9Omnnzo9JjcyYcIExcfHy9fXV2+//bY2bdqkd955R35+fjp27JhmzpyZZZ9//vlHly9f1ocffqivv/5aTZo0kZT+I9i6deuMepMmTTIS6L1799aGDRsUEREhs9lsM7lLcvw76sSJExo+fLi+/fZbjR8/3niIcEYfS9KqVaskSbVr19bKlSu1efNmzZo1S/fcc488PT31zTff5LS7AIeQRAdcJPNs1OxkJG6uXbumnTt36uzZs6pbt66WLVumvXv3avny5fLw8JCfn5/NL+MZa9Fl98CW1157TVWqVNGTTz55w5kgmY0bN05NmjRRrVq1NHHiRCOI+/HHH296DtkpWbKkTdtKlChxw/UDT548aSQiH3nkEY0aNUq1atXSgw8+qDlz5hgPiMqYhZyZyWTSu+++q6CgINWrV089e/Y0yjInsrOTEaT4+/srIiLCuP1u2rRpqlWrlqT0AOmff/6Rv7+//P39jX0LFSpkdwwk2STXbmX9tzfeeEP169dXjRo1bJKMGbPcd+7cqWPHjkmSOnXqpAEDBqhatWpq0aKFceFhtVqNc834zEnSTz/9pOjoaFWuXFnz5s3T3r17tWnTJuOhUxs2bJAkeXt76+WXX1aRIkVUunRpYxbA+fPns72tr1mzZurXr58qVaqkHj16qHr16pLSZ2TcCk9PT7Vv316LFy82Lmyk9FszM2YOvfjiiypbtqwKFSqkAQMGyM/PT1ar1bjt0pnzz6xSpUqaOHGiatWqpaZNm2rcuHFGWUZgvGrVKuM2zLFjx+qZZ55R1apV1a1bN7388suS0mcDZXdB6kifZbT90qVL2rVrlxITE9WoUSNt2LBBUVFRmjVrlqT08c640AgJCVGbNm3k5+en2rVrq3379pLSb5e9Pri9FRmflcDAQHXv3l333HOPKlWqZNyGfejQoWxnB3bq1EmdOnVSxYoVNXToUOPuisznnXELr5+fn/FvPTQ0VDNnzjS+GzIULVpURYoUMV5nfGavf/hX4cKF9dZbb6l69epq1KiRzZqocXFxkhz/fgYAuA9i73R3Y+ztiCeeeEIRERE2Se3//ve/xnJuzz//vKpVq6aCBQuqc+fOxuzhzMtQXM+ZuCstLc1I8FWqVEljxoxRjRo11LhxY7311lsaPXq0IiIi5O3tfcMxdSbuylgipmDBgnr33XdVt25d1a9f/6az651RrFgxvfDCC/riiy9sfkhy5lrC0Tj3es8++6x69+6tqlWrqmPHjsbn8erVq/rvf/8rSVq6dKnRjjlz5ujBBx9UzZo1FR4erubNm0tKn0iR3ZJE/fv3V5s2bVSpUiWb5Zkyx6oZE6aio6O1b98+WSwWdezYUTt27FBkZKT69OkjKX1Szp9//ilJeu655xQUFCQ/Pz+1bNnSuMP2Rp81Z126dEk//vijpPTlUx588EEVKFBADzzwgB5//HFJ0pdffpntteqIESP0yCOPqGrVqsZzBDKf94ULF7R3715J6Z/7ESNGqHr16mrZsqXGjh2b5f0c/Y56+OGH9cILL6hixYrq3LmzatSoIen/43Pp/z8rZ8+eVWRkpK5evapWrVrp+++/1969e+1O3gFyC8u5ALksY6Zp5oRddrp3764ffvhBBw8e1EcffaSPPvpI3t7eCgoKUqtWrdSpU6dsE3r2FCxY0Jgh64x69eoZf/fy8lKdOnUUGxur5ORkXbx48YZBeG48ICTzbNKHHnrIpiwgIEBVq1bVwYMHbW7Vy3B9gJ357ze6CLl48aLxP+Pg4GAjeSdJHh4eaty4sQ4dOqSUlBT99ddfCgoKcuqcMrfj/PnzduulpaXdMBGXeR3MzJ+FjKTxjfqubt26Klq0qC5dumT0Xdu2bfXVV19px44dWrFihVasWCFPT0/VqlVLLVq0UJcuXYyLxoxZMikpKTbrgmb2+++/q2XLljbbMmaIZ8gILJ25KCxdurSWLVumhIQEffjhh/r2229lsVh0+vTpLA8qynw77ODBg7N9v4yZ6M6cf2ZBQUE2idjMtwhmzOLK/Pm8fiwyv87uc+xInw0cOFC//PKLTp06pVmzZmnWrFny8/NTaGionnjiCYWFhcnHx0cXL15UYmKipPR1Xx999NEsx0tLS9OhQ4eM21BvVcYYnD59OtvjSemflfvuu89mW+bzNplMKlasmP755x+b885YS75y5co236klSpRQpUqVsu3Pm6lcubJ8fHyM15lvY834t5Xb388AANch9nbOnRh7X2/r1q26dOmSli1bps8//1xSepLz+vfNHEe+9dZbWZaJkP4/jsyOM3FXjRo1jHWoa9asacyylaTGjRurcePGDp2bM3HX6dOnJaXHPpknGlSqVEnFihUz2u6o1q1ba/jw4Tpx4oSmTJmiP//8U5cuXVJaWprNMi6Z2+nItYSjce71QkNDbV5nF6NnfN4rV66c5e7Ehx56SFu2bJGUHqNfv1Rg5ljV399fJpNJVqvV5rM+ZswYDRkyRNHR0XrzzTeNuo0aNVK7du2MJWsyJ+k//vhjffzxx1nO58iRIzKbzdmeq7NOnjxpTOz6+uuvs53Nf+nSJcXExGQZu2rVqhl/z+7feOZnPYWEhNjsm93SlY66/hkQGTF65qWWhg8frr59++r8+fPGnSuFCxdWgwYN9NRTT+mpp55iogtcik8XkIuio6ON25oyZlTYU6xYMa1atUpz5sxRp06dVKlSJaWkpGjPnj2aMmWK2rdvn+WBHzeS04TO9QFv5v/pXD+DU5LNgz1y4wEo2R0jsxtdLFz/cElH/4d5s5lCmY+ZOcB1VPny5Y1E6IEDB7Jd9uTChQt68MEH9eqrrxqzma+X+fwyn1tG+27WdxkyzsHHx0cLFizQ4sWL1b17d9WoUUNWq1UHDhzQ+++/r7Zt2yo2Ntbh977+QYzXt9mZNl6/T2BgoOrVq6cZM2aoYcOGktLXcBw8eLDNTH9n2unM+Wd2/fhl9/lwdiwyc6TPypcvr2+++UbvvvuunnnmGQUGBiopKUk//fST3njjDfXo0UMpKSkOtyO7scspR46Z3cx3R847Y13I3Ega2Dtudv+2cvv7GQDgGsTezrsTY+/rlSlTRrVq1dK4ceOMZTj++OMPvfDCCzZrODty98CNYiZn4q7M55iTOw6cOWZG3JXRl9mNaU4SjX5+fipXrpwefvhhLViwQMWLF5fVatXHH3+siIgIp9uZ0beOxrnXcyRGv9EY3+xz58jnvXnz5vr+++81ZswYNWvWTEWLFtWFCxf0zTffqF+/fpo6depN25HBYrHcdFkqRzl6Z0x2n+/Md3veKD6X8i5Gz1CvXj1t3rxZkyZN0hNPPKESJUro77//1pYtWzRs2DCbmfOAK5BEB3JR5nXCMm6Tsuf48ePatm2bkpKSNGHCBG3atEk///yzsWzHiRMnjFvwMv9P3d7/qHL6EJLM6xdaLBYdPHhQUvovuhkzejL/Gp55TefMD6nJzJH2Zsj8C//PP/9sU3b27FnjIT3Xz2K9FYULFzZm+ezfv99mTcC0tDTt2rVLUvp5V6lSxen3N5lMeuaZZySlPxhx2bJlWeosWrRIiYmJ+vrrr41xdtaN+u733383grCMi8rY2Fht375dMTExGj16tNavX69ffvnFuO3t4sWLWr9+vSQZt7D6+fnp999/1+HDh3X48GH99NNP+vLLL/Xbb79p/PjxOWq3Mzw9PTVlyhTjQvWnn37SkiVLjPKMdkrSsmXLjHYeOHBAK1asUGRkpDZu3Oj0+WcWFRVlE6T/9ttvWY5/o7H46aefjL/f7AI/O2lpaTpy5Ii2bt2qggULatq0adqyZYu2b9+udu3aSUqf/fTbb7+paNGixr/bJk2aGP1x+PBhbdq0SZs3b9Yff/yhp556yul22JPRBxkzwzP+27p1q7755hsdOHBAffv2zdF7ly9fXlL692Hm2VLnzp3L9rZbZ757bsSZ72cAQP4h9pbD7c1wJ8beNzJ27FgFBgZKSl/TPfNSJpln4L7zzjs2cczKlSu1a9cum7jves7EXf7+/sbM2kOHDtn8OLJ582Z169ZNo0ePNpZqtDemzsRdGed3/Phxm8/RiRMnbnlCRalSpWyuBT744INsY+SbXUs4E+deb/fu3TavbxSjHz9+3GZZEMn285+Tz3tycrL279+vH3/8Uffff7/mzZunXbt26auvvjJi/k8//VQWi8Xms/bqq6/ajN3atWu1Y8cOHT58+IZ3ojgj8/E6depkc7wNGzZo69atOnz4cJaZ5I7IiM8lad++fTZlGf+Wr5cbMXpKSor++OMPbd++XRUqVNCsWbP03//+V5s3bzaec7Bx40Zj6VPAFVjOBcihhIQEeXl5KS0tTZcvX9YPP/ygefPmSZLKlStn/E/fnnHjxunnn3+Wh4eH4uPj1axZMyUnJ9t86Wf8gpz51+CTJ0/q2LFjKliwYJZb0nJizJgxslgsqly5spYsWaLo6GhJUsuWLY1ff0uVKmXUX7JkiQYOHKjDhw9r/vz52b5n5sB///79SkxMtHtbZuXKlVW/fn1FRkZq27Ztmjx5ssLCwpSYmKipU6caM26ef/75Wz7XzDp06KA5c+bo/PnzevnllzV48GD5+Pho0aJFxm1/7du3V8GCBXP0/v379zf+Jz5lyhRduHBBTz31lNLS0rR+/XrjFr4CBQrkOMH44IMPqmzZsoqNjdXy5csVEBCgli1b6tSpU8btqCaTyei7Dz74wHgYS3R0tPEZzXxBlvGZe/rpp/Xtt98qKSlJr732ml588UV5enpq8uTJ+vnnn+Xp6alVq1ZlWV7FFQIDAzVy5EiNHj1aUvoDnFq1aqWyZcvq8ccf17Rp03Tt2jWNHz9eI0eOVJkyZbRs2TItXLhQkjRq1Cj16NHDqfPPLC4uTsOHD1f//v116dIl41ZNSWrVqpWk9DUZFyxYoNTUVI0fP14Wi0X33XeffvnlF73//vuS0pcgcfSBWNcbNGiQTpw4oQIFCujNN99U/fr1lZiYaLMmY+ax++STT/TTTz9p7ty5evzxx3X+/HkNGzZMcXFxCggI0Lfffptltkd2EhMTs137XkqfgXf//ffr6aef1oEDB3TixAlNnDhRnTt3VnJyskaNGqXDhw+rYMGC2rRpk833iKOeeuop/fHHH0pKStLQoUM1ZMgQJSUladq0adne4ZH5u/KPP/7QH3/8oYoVKzr979iZ72cAQN4g9ib2zolChQpp4sSJ6t27t6T0pObTTz+t4OBgPfjggypRooQSEhL03nvvqVChQqpatao2b96sKVOmSJL+/e9/26yHfT1n4q7WrVtr5cqVio2N1dixY9WjRw/9/fffmjZtmk6cOKGDBw8aD7m0N6bOxF2PP/64jhw5oqSkJA0ePFivvPKKUlJSNGnSpFzp25YtW+rpp5/W+vXrZbFYNHr0aK1evVpeXl4OX0vUqlXLqTg3s02bNikiIkKtW7fWb7/9pkWLFklKH/OMpYrat2+vvXv3KiUlRQMGDNBrr70mf39/rV271ljK5ZFHHsmypIkjEhIS1KVLF1ksFlWsWFFvvPGGKlSooLNnz+rvv/+WlD6b2mQyqUaNGqpVq5YOHTqkjz/+WOXLl1dQUJD27t2r8PBwWSwWPf7448Z1w82cPHnSboxetmxZVa1aVc2bN9eWLVu0du1a1alTR40bN9bx48c1bNgwXblyRXXq1NHq1audvvujcOHCeuSRR7R9+3ZFRUUZd/IcO3ZMEyZMyHYfZ76j7DGbzerRo4cuX74sf39/TZgwQTVr1tTZs2dt7r7IyZ3QgKO4AgRyqHPnztluL1asmN5///2brmUWHh6unj176uLFi5o8ebImT55sU167dm1j/Th/f3/jCe979+7Vk08+qeHDh+uFF164pXMIDAxUSkqK8eDDDCVLlrS5Fapt27bGU8hnz56tiIgIWa1WhYSEZLv8RebZtkOGDJGfn1+WX6kzmzJlip5//nnFxcVp0aJFRgCUoWfPnnryySdzcIb2DRgwQHv37tXOnTv13//+13j4TIb777/f7pPaHeHv768PP/xQ/fv31+nTpzV37lzNnTvXpo6Pj4/eeecdm1/zneHp6akZM2bohRde0KVLl/Tee+/pvffeM8pNJpNGjhyp4OBgSelrhu/atUsxMTGaP39+lguxwMBAhYWFSUpPDj/66KPatm1btuvodejQIU8S6Bk6duyojRs3aseOHUpKStL48eM1b948BQQEaNCgQZo+fbr++OMPmwdcSVKdOnWMBzs5c/6ZBQQEaNOmTcaM9gxhYWFq1KiRpPRZLqNHj9aECRN04cIFDR061KZuoUKFNHPmzBzd+u3h4aE333xT/fv3V3JycrYXcs2aNTPWWO3fv79++OEHxcbGZvlMeHh4aOjQoQ4l0KX0GVuZH2qbWa1atbRu3Tp17dpV69ev14EDB/TJJ59keRBZv379cpRAl6Ru3bppzZo1+uuvv7Rjxw7jQWhFihQxvhMzy7yWYkZbVq5cabP+rCOc+X4GAOQNYm9i75xq0qSJOnfurOXLlystLU1jx47V6tWr5efnp/DwcA0fPlyxsbHq37+/zX5ly5bVv//97xu+tzNx15AhQ7R7926dPHlSq1atMiZ3ZBg1apQxW93emDoTd/Xp00ffffed/vzzT5s+9/PzU+XKlbO9q89Zo0eP1s6dOxUfH69Dhw5p8eLF6tOnj1PXEs7EuZmVLl3aWEM9g8lk0qhRo4yYu0OHDtq9e7e+/PJLHThwIMu1QpUqVbJ8FziqXLly+s9//qN3331XJ0+eNH6oyWzgwIHGj2OjRo0yrtuGDBliU69YsWIaNGiQw8dev359tnfQSlKPHj00atQoDR8+XHv37lViYqLeeOMNmzoFChTQ8OHDc7x80tChQ/XLL78oKSnJZo33qlWrGvF55vd29jsqO/fcc4/GjBmjESNG6MKFCxo4cGCWOl27ds32GVdAbmE5FyAX+Pn5qXr16urTp482bNjgUHKxZs2aWrVqlZ5//nlVqlRJfn5+8vX1VbVq1dS/f3998sknxsWAh4eHxo8fr5o1a8rX11clS5a0eThMTt17771avny5nnjiCRUqVEiFCxfWk08+qRUrViggIMCo16RJE02bNk01atSQj4+PAgMDNXDgQM2ZMyfb93322Wf19NNPq3jx4vLz81O1atWUnJxstx3ly5fXhg0bNGDAAFWvXl0FChTQPffco0aNGmn27Nl6/fXXb/lcr+fj46OFCxdqwoQJql+/vgoXLixfX1/VqlVLI0aM0OLFi22eIp4TNWvW1Pr16/Xqq6+qTp06KliwoLy9vVWuXDl16tRJX375pTGTOafq1aunDRs2qGfPnqpUqZJ8fHxUtGhRNWvWTIsXL1avXr2MugEBAVqxYoX69eunatWq6Z577pG3t7cqVKig559/XitXrjQeHmMymTR79myNGDFC9913n/z8/FSoUCHVrVtXEyZM0Lhx426p3TkxceJE48Ji69atRlK7b9++ioiIUKNGjVSkSBH5+vqqUqVK6t+/v5YsWWI8vMqZ88/soYce0uzZs1WjRg15e3urfPnyGjp0aJZZPF27dtXy5cvVtm1blSpVSt7e3goICFD79u31xRdf3NKDdh588EGtXLlS7dq1U/ny5eXj4yM/Pz/dd999GjZsmM2slRIlSmjlypXq0aOHypcvL29vb/n7++vhhx/WwoULjbVBc0uBAgW0ZMkSDRgwQFWrVpWvr6+KFi2qBg0aaNasWVkuSJ1xzz33aOnSpQoLC1OxYsVUoEABNWnSRJ9++qlxgZg5aVK1alX95z//Ubly5eTj46NKlSrl6LjOfD8DAPIesbctYu+be+2111S2bFlJMpK9UvoPFosXL1azZs1UrFgxI1bv3r27cafnjTgTd/n7+2vlypXq3bu3KlSoIG9vb5UoUUJNmjTRggUL1LFjR6OuvTF1Ju6655579Mknn6hz586699575efnZ8RRNWrUyJV+LVasmM11wezZsxUbG+vUtYQzcW5mgwYN0quvvqrSpUvLx8dHderUUUREhM2kGJPJpGnTpmnGjBlq0qSJMcaVK1fWgAEDtHLlyltKuvbr10/z589X06ZNFRAQIC8vLxUpUkSNGjUyJjtlaNiwoVasWKE2bdqoRIkS8vb2VpkyZRQWFqYVK1ZkebDmrapatapWrVqlsLAwlS5dWt7e3ipZsqSeeOIJffbZZ3rwwQdz/N61atXSp59+qoceekgFCxZU0aJF1aFDB3300UdGncxLXjn7HWXPM888o08//VSPP/64AgMD5e3trXvuuUchISEaP368xowZk+NzAhxhsubmkwAAALgDnDp1ypiN9txzzxlPf0feioyMlNlsVpkyZRQYGGiTuH7iiSd04sQJhYSEaPny5fnYSgAAcDd46KGHdP78eVWqVEmbNm3K7+bcldasWaPw8HBJMpYiQt77/vvvVaxYMZUpU0alS5c2ZtufO3dOTZs2lZQ+0en6u3OB2x3LuQAAALf09ddfa+nSpZKk5s2bG7edbt26VSdOnJAkp9dTBAAAcMYPP/yg//73vzp//ryk9DsKgLvZjBkz9Oeff0pKX7Lmueee05UrV4xnUknE6LgzkUQHAABuqVOnTlq1apWSk5O1ZcsW4wFQGQoVKpRlbUsAAIDcNGXKFOPHe0m3tAwGcCfo0aOHRo8eLUmKiIhQRESETfl9992nFi1a5EfTAJciiQ4AANxSzZo1tWzZMs2fP1979+41ZoCVKFFCDRs2VP/+/XP8YF4AAICbMZvNktLXdy5cuLDatGlzS897Ae4EHTt2VJEiRbRs2TIdOnRIly9flpeXl8qWLavmzZvrpZdekpcX6UbceVgTHQAAAAAAAAAAOzzyuwEAAAAAAAAAALgrkugAAAAAAAAAANhBEh0AAAAAAAAAADtY6f8G4uP/zpfj+vh4ymy25MuxcWOMjftibNwXY+O+GBv3xdi4pxuNS8mShfO4NXcH4vE7F33sevSx69HHrkcfux597Hr0sev5+HiqaNGCLj0GM9HdjMlk+yfcB2Pjvhgb98XYuC/Gxn0xNu6Jcbl7MNauRx+7Hn3sevSx69HHrkcfux597Hp51bck0QEAAAAAAAAAsIMkOgAAAAAAAAAAdpBEBwAAAAAAAADADpLoAAAAAAAAAADYQRIdAAAAAAAAAAA7SKIDAAAAAAAAAGAHSXQAAAAAAAAAAOwgiQ4AAAAAAAAAgB0k0QEAAAAAAAAAsIMkOgAAAAAAAAAAdpBEBwAAAAAAAADADpLoAAAAAAAAAADYQRIdAAAAAAAAAAA7SKIDAAAAAAAAAGAHSXQAAAAAAAAAAOwgiQ4AAAAAAAAAgB0k0QEAAAAAAAAAsIMkOgAAAAAAAAAAdpBEBwAAAAAAAADADpLoAAAAAAAAAADYQRIdAAAAAAAAAAA7vPK7AbCVmpqq6OjjSkmxOLxPhQqV5OXFUAIAAAC3ingcAAAA1yPSczMnT57Ql3+ckH9AOYfqJ5w5pXaSqlSp5tJ2AQAAAHeDkydP6OyaFQr095esN69/6sJ5qX1n4nEAAIA7GEl0N1S8TDkFVKia380AAAAA7krlihdX5YAAh5LokuT4nHUAAADcjlgTHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHaQRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHbkexL97NmzGjx4sBo2bKimTZtq8uTJunbtmiRp4sSJqlmzps1/S5cuNfbdsGGDWrZsqeDgYA0cOFAXLlwwyqxWq9555x01btxYDRs21NSpU5WWlpbn5wcAAAAAAAAAuH155efBrVarBg8erCJFiujTTz/VpUuX9Prrr8vDw0MjRozQsWPHNHToUD333HPGPoUKFZIk7d+/X6NGjdK4ceNUq1YtTZo0SeHh4Zo3b54kaeHChdqwYYNmz56t1NRUDR8+XMWLF1efPn3y5VwBAAAAAAAAALeffJ2J/tdff2nfvn2aPHmyqlevrgYNGmjw4MHasGGDJOnYsWOqXbu2SpYsafzn5+cnSVq6dKnatGmjdu3aqVatWpo6daq2bdummJgYSdKSJUs0ePBgNWjQQI0bN9awYcP06aef5tu5AgAAAAAAAABuP/maRC9ZsqQ++ugjlShRwmb7lStXdOXKFZ09e1aVKlXKdt+oqCg1aNDAeF2mTBkFBgYqKipKZ8+e1ZkzZ/TAAw8Y5fXr11dsbKzOnTvnknMBAAAAbmdms1lt27bVrl27jG2nT5/Wiy++qODgYLVq1Upff/21zT4srwgAAIC7Qb4m0YsUKaKmTZsar9PS0rR06VI1btxYx44dk8lk0ty5c/XII4/omWee0dq1a426586dU6lSpWzer3jx4oqLi1N8fLwk2ZRnJOrj4uJceUoAAADAbefatWsaMmSIjhw5YmxLTU1Vv3795OXlpbVr16pPnz567bXX9Oeff0r6/+UVBw0apOXLl+vy5csKDw839s+8vOKsWbO0fv16LVy4MM/PDQAAALhV+bom+vWmTZumgwcPatWqVTpw4IBMJpOqVKmi559/Xr/88ovGjBmjQoUKqVWrVkpOTpaPj4/N/j4+PjKbzUpOTjZeZy6T0mfYOMNkusWTclLG8UySrDnYD65jjA197XYYG/fF2LgvxsZ9MTbu6U4el6NHj2ro0KGyWm2jz23btunMmTNatmyZChUqpCpVqmj79u3au3evatSoYbO8oiRNnTpVzZs3V0xMjMqXL2+zvKIkDRs2TDNnzuQZRQAAALjtuE0Sfdq0aVq8eLHee+891ahRQ9WrV1fz5s1VrFgxSVKtWrV04sQJLVu2TK1atZKvr2+WhLjZbJafn59NwtzX19f4uyRjTXVH+Ph45sKZOcfb21Mmk0Umj/RE+s2YPNL38fbO+7bebUwmydPTUyaTZHXmFw64HGPjvhgb98XYuC/Gxj3dyeOye/duNWrUSK+++qpCQkJstj/44IMqVKiQse2DDz4w/h4VFaUXX3zReJ15eUUfH58bLq94/R2lAAAAgDtziyT6hAkTtGzZMk2bNk1PPPGEJMlkMhkJ9AxVqlTRzp07JUkBAQFKSEiwKU9ISFDJkiUVEBAgSYqPj1e5cuWMv0vp67A7ymy25Plso5QUi6xWq6xpjs1Et6al75OSYnF52+52GRfNqamWO+7i+XbH2LgvxsZ9MTbui7FxT3fyuHTr1i3b7TExMSpbtqzeeecdrVu3Tvfee68GDx6sli1bSrq15RWdSaLn152h6S8c2SGb/XBDd/KdHe6CPnY9+tj16GPXo49djz52vbzq23xPos+ePVuff/653n33XbVu3drYPnPmTO3du1eLFi0yth06dEhVqlSRJAUHBysyMlJhYWGSpDNnzujMmTMKDg5WQECAAgMDFRkZaSTRIyMjFRgY6PSsl7y+SMo4nrOHvdMu5tyZ1Up/uyvGxn0xNu6LsXFfjI17upvG5erVq1q7dq2efPJJzZ07V7t27dLgwYO1fPlyBQUF5cnyivl1Z6hMkoeDV2QeJpM8uDPUKXfynR3ugj52PfrY9ehj16OPXY8+dr2MPna1fE2iHzt2TB988IH69u2r+vXrGzNWJKl58+aaP3++FixYoFatWmnHjh364osvtGTJEklS165d1b17d4WEhCgoKEiTJk1Ss2bNVL58eaP8nXfeUenSpSVJ06dPV+/evfP+JAEAAIDbkKenp4oVK6Y333xTHh4eqlOnjn799VetWLFCQUFBebK8Yn7dGeplldIcvNJNs1pl4c5Qp9zJd3a4C/rY9ehj16OPXY8+dj362PXy6geKfE2if//997JYLJozZ47mzJljU3b48GHNnDlTs2bN0syZM1W2bFlNnz5doaGhkqTQ0FCNHz9es2bN0qVLl9SkSRNNmDDB2L9Pnz46f/68Bg0aJE9PT3Xo0EG9evXKy9MDAAAAblulSpWSyWSSh4eHsa1y5co6fPiwpLxZXlHKvztD0184skM2+8Ehd9OdHfmFPnY9+tj16GPXo49djz6+/eVrEr1v377q27ev3fKWLVsaay5mJywszFjO5Xqenp4KDw9XeHj4LbcTAAAAuNsEBwdrzpw5slgsxi2yx44dU9myZY3yvFheEQAAAMhvHjevAgAAAOBu07ZtW6WlpWncuHE6efKkPv30U/3444/q1KmTpPTlE9etW6eVK1fq0KFDeu2117JdXnHXrl3atWuXpk+frh49euTnKQEAAAA5ku8PFgUAAADgfgoVKqSFCxfqzTffVNu2bRUYGKj33ntPderUkcTyigAAALh7kEQHAAAAIEnGeucZqlWrpqVLl9qtz/KKAAAAuBuwnAsAAAAAAAAAAHaQRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHaQRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHaQRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHaQRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHaQRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHaQRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO7zyuwEAAAAAcLtKtVgUEx3t1D4VKlSSlxeXYgAAALcLIjcAAAAAMpvNCgsL05gxY9SoUSObsr///ltPPvmkXn31VYWFhRnbN2zYoBkzZig+Pl4PP/ywJkyYIH9/f0mS1WrV9OnTtWrVKqWlpalDhw4aNmyYPDzurJthT1+8oLQft8gzsKxD9U9dOC+176wqVaq5uGUAAADILSTRAQAAgLvctWvXNHToUB05ciTb8mnTpuncuXM22/bv369Ro0Zp3LhxqlWrliZNmqTw8HDNmzdPkrRw4UJt2LBBs2fPVmpqqoYPH67ixYurT58+Lj+fvBbo76+qpUs7XN/iwrYAAAAg991Z00AAAAAAOOXo0aPq1KmTou0sSfLrr79q586dKlmypM32pUuXqk2bNmrXrp1q1aqlqVOnatu2bYqJiZEkLVmyRIMHD1aDBg3UuHFjDRs2TJ9++qnLzwcAAADIbSTRAQAAgLvY7t271ahRIy1fvjxLmdls1pgxYzR27Fj5+PjYlEVFRalBgwbG6zJlyigwMFBRUVE6e/aszpw5owceeMAor1+/vmJjY7PMaAcAAADcHcu5AAAAAHexbt262S2bO3euateurYcffjhL2blz51SqVCmbbcWLF1dcXJzi4+Mlyaa8RIkSkqS4uLgs+92IyeRw1VxhczxHjm267k8H6+f1ebkTE33gcvSx69HHrkcfux597Hr0sevlVd+SRAcAAACQxdGjR/X555/ryy+/zLY8OTk5y+x0Hx8fmc1mJScnG68zl0nps9sd5ePj6Wyzb5m3t6dkkjwcvCLzMJlk8nCuvoe3Z/px7lImk+Tp6SmTSbJa87s1dyb62PXoY9ejj12PPnY9+tj1MvrY1UiiAwAAALBhtVo1evRoDR482JhBfj1fX98sCXGz2Sw/Pz+bhLmvr6/xd0ny8/NzuB1msyXPZ26lpFjkZZXSHLzSTbNaZU1zrr4lxaKUlLv38aIZiYTUVAsJBRehj12PPnY9+tj16GPXo49dL69+oCCJDgAAAMDG6dOntXfvXh0+fFhTpkyRJCUlJemNN97Q119/rY8++kgBAQFKSEiw2S8hIUElS5ZUQECAJCk+Pl7lypUz/i4pywNKbyavLzhtjufIsa3X/elgfS6k0/uAfnAt+tj16GPXo49djz52Pfr49kcSHQAAAICNgIAAffvttzbbunfvru7du+uZZ56RJAUHBysyMlJhYWGSpDNnzujMmTMKDg5WQECAAgMDFRkZaSTRIyMjFRgY6NR66AAAAIA7IIkOAAAAwIaXl5cqVqyYZVvx4sWNWeZdu3ZV9+7dFRISoqCgIE2aNEnNmjVT+fLljfJ33nlHpUuXliRNnz5dvXv3ztsTAQAAAHIBSXQAAAAATgsNDdX48eM1a9YsXbp0SU2aNNGECROM8j59+uj8+fMaNGiQPD091aFDB/Xq1Sv/GgwAAADkEEl0AAAAAJKkw4cP2y374YcfsmwLCwszlnO5nqenp8LDwxUeHp5r7QMAAADyg0d+NwAAAAAAAAAAAHdFEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7Mj3JPrZs2c1ePBgNWzYUE2bNtXkyZN17do1SVJMTIx69eqlkJAQPfnkk9qxY4fNvj/99JPatm2r4OBg9ejRQzExMTblixYtUtOmTRUaGqrXX39dSUlJeXZeAAAAAAAAAIDbX74m0a1WqwYPHqykpCR9+umneu+997RlyxbNmDFDVqtVAwcOVIkSJbR69Wo9++yzGjRokE6fPi1JOn36tAYOHKiwsDCtWrVK/v7+GjBggKxWqyRp06ZNmj17tsaPH6/FixcrKipK06ZNy8/TBQAAAAAAAADcZvI1if7XX39p3759mjx5sqpXr64GDRpo8ODB2rBhg3bu3KmYmBiNHz9eVatWVb9+/RQSEqLVq1dLklauXKm6deuqd+/eql69uiZPnqzY2Fjt3r1bkrRkyRL17NlTzZs3V7169TRu3DitXr2a2egAAAAAAAAAAIflaxK9ZMmS+uijj1SiRAmb7VeuXFFUVJRq166tggULGtvr16+vffv2SZKioqLUoEEDo8zPz0916tTRvn37ZLFY9Ntvv9mUh4SEKCUlRYcOHXLtSQEAAAAAAAAA7hhe+XnwIkWKqGnTpsbrtLQ0LV26VI0bN1Z8fLxKlSplU7948eKKi4uTpBuWX758WdeuXbMp9/LyUrFixYz9HWUyOXtWtybjeCZJ1hzsB9cxxoa+djuMjftibNwXY+O+GBv3xLgAAAAAd698TaJfb9q0aTp48KBWrVqlRYsWycfHx6bcx8dHZrNZkpSUlGS3PDk52Xhtb39H+Ph45uQ0bom3t6dMJotMHumJ9JsxeaTv4+2d922925hMkqenp0wmyerMLxxwOcbGfTE27ouxcV+MjXtiXAAAAIC7l9sk0adNm6bFixfrvffeU40aNeTr66vExESbOmazWQUKFJAk+fr6ZkmIm81mFSlSRL6+vsbr68v9/PwcbpPZbMnz2UYpKRZZrVZZ0xybiW5NS98nJcXi8rbd7TIumlNTLVw8uxnGxn0xNu6LsXFfjI17YlwAAACAu5dbJNEnTJigZcuWadq0aXriiSckSQEBATp69KhNvYSEBGOJloCAACUkJGQpv++++1SsWDH5+voqISFBVatWlSSlpqYqMTFRJUuWdKpteX2RlHE8Zw/LxVzesVrpb3fF2LgvxsZ9MTbui7FxT4wLAAAAcPfJ1weLStLs2bP1+eef691339VTTz1lbA8ODtaBAweMpVkkKTIyUsHBwUZ5ZGSkUZaUlKSDBw8qODhYHh4eCgoKsinft2+fvLy8VKtWrTw4KwAAAAAAAADAnSBfk+jHjh3TBx98oBdffFH169dXfHy88V/Dhg1VpkwZhYeH68iRI5o/f77279+vDh06SJLat2+vPXv2aP78+Tpy5IjCw8NVrlw5NWrUSJLUrVs3LViwQJs3b9b+/fv15ptvqlOnTk4t5wIAAAAAAAAAuLvl63Iu33//vSwWi+bMmaM5c+bYlB0+fFgffPCBRo0apbCwMFWsWFEREREKDAyUJJUrV07vv/++3nrrLUVERCg0NFQREREy/W8R86eeekqxsbEaO3aszGazHn/8cQ0fPjzPzxEAAAAAAAAAcPvK1yR637591bdvX7vlFStW1NKlS+2WP/roo3r00Udz/P4AAAAAAAAAANxIvq+JDgAAAAAAAACAuyKJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAZDab1bZtW+3atcvYtm/fPnXp0kWhoaF64okntHLlSpt9fvrpJ7Vt21bBwcHq0aOHYmJibMoXLVqkpk2bKjQ0VK+//rqSkpLy5FwAAACA3EQSHQAAALjLXbt2TUOGDNGRI0eMbfHx8XrxxRfVsGFDrV27VoMHD9aECRO0detWSdLp06c1cOBAhYWFadWqVfL399eAAQNktVolSZs2bdLs2bM1fvx4LV68WFFRUZo2bVp+nB4AAABwS0iiAwAAAHexo0ePqlOnToqOjrbZvnnzZpUoUUJDhgxRpUqV9NRTT6ldu3Zav369JGnlypWqW7euevfurerVq2vy5MmKjY3V7t27JUlLlixRz5491bx5c9WrV0/jxo3T6tWrmY0OAACA2w5JdAAAAOAutnv3bjVq1EjLly+32d60aVNNnjw5S/0rV65IkqKiotSgQQNju5+fn+rUqaN9+/bJYrHot99+sykPCQlRSkqKDh065KIzAQAAAFzDK78bAAAAACD/dOvWLdvt5cqVU7ly5YzX58+f11dffaWXX35ZUvpyL6VKlbLZp3jx4oqLi9Ply5d17do1m3IvLy8VK1ZMcXFxTrXPZHKq+i2zOZ4jxzZd96eD9fP6vNyJiT5wOfrY9ehj16OPXY8+dj362PXyqm9JogMAAAC4oeTkZL388ssqUaKEOnfuLElKSkqSj4+PTT0fHx+ZzWYlJycbr7Mrd5SPj+ctttx53t6ekknycPCKzMNkksnDufoe3p7px7lLmUySp6enTCbpf0voI5fRx65HH7sefex69LHr0ceul9HHrkYSHQAAAIBd//zzjwYMGKATJ07os88+k5+fnyTJ19c3S0LcbDarSJEi8vX1NV5fX56xvyPMZkuez9xKSbHIyyqlOXilm2a1yprmXH1LikUpKZZbaeZtLSORkJpqIaHgIvSx69HHrkcfux597Hr0sevl1Q8UJNEBAAAAZOvKlSt64YUXFB0drcWLF6tSpUpGWUBAgBISEmzqJyQk6L777lOxYsXk6+urhIQEVa1aVZKUmpqqxMRElSxZ0qk25PUFp83xHDm29bo/HazPhXR6H9APrkUfux597Hr0sevRx65HH9/+eLAoAAAAgCzS0tI0aNAgnTp1Sp988omqV69uUx4cHKzIyEjjdVJSkg4ePKjg4GB5eHgoKCjIpnzfvn3y8vJSrVq18uwcAAAAgNxAEh0AAABAFqtWrdKuXbs0ceJEFSlSRPHx8YqPj1diYqIkqX379tqzZ4/mz5+vI0eOKDw8XOXKlVOjRo0kpT+wdMGCBdq8ebP279+vN998U506dXJqORcAAADAHbCcCwAAAIAsNm3apLS0NPXr189me8OGDfXJJ5+oXLlyev/99/XWW28pIiJCoaGhioiIkOl/i5g/9dRTio2N1dixY2U2m/X4449r+PDh+XEqAAAAwC0hiQ4AAABAknT48GHj7wsWLLhp/UcffVSPPvqo3fK+ffuqb9++udI2AAAAIL+wnAsAAAAAAAAAAHaQRAcAAAAAAAAAwA6WcwEAAACAPJJqsSgmOtqpfSpUqCQvLy7dAAAA8guRGAAAAADkkdMXLyjtxy3yDCzrUP1TF85L7TurSpVqLm4ZAAAA7CGJDgAAAAB5KNDfX1VLl3a4vsWFbQEAAMDNsSY6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAMhsNqtt27batWuXsS0mJka9evVSSEiInnzySe3YscNmn59++klt27ZVcHCwevTooZiYGJvyRYsWqWnTpgoNDdXrr7+upKSkPDkXAAAAIDeRRAcAAADucteuXdOQIUN05MgRY5vVatXAgQNVokQJrV69Ws8++6wGDRqk06dPS5JOnz6tgQMHKiwsTKtWrZK/v78GDBggq9UqSdq0aZNmz56t8ePHa/HixYqKitK0adPy5fwAAACAW5GjJPqGDRtkNptzuy0AAAAAHJRbMfnRo0fVqVMnRUdH22zfuXOnYmJiNH78eFWtWlX9+vVTSEiIVq9eLUlauXKl6tatq969e6t69eqaPHmyYmNjtXv3bknSkiVL1LNnTzVv3lz16tXTuHHjtHr1amajAwAA4LaToyT6a6+9piZNmujNN9/U/v37c7tNAAAAAG4it2Ly3bt3q1GjRlq+fLnN9qioKNWuXVsFCxY0ttWvX1/79u0zyhs0aGCU+fn5qU6dOtq3b58sFot+++03m/KQkBClpKTo0KFDOW4rAAAAkB+8crLTDz/8oLVr12rdunVavny5KleurPbt2+uZZ55RyZIlc7uNAAAAAK6TWzF5t27dst0eHx+vUqVK2WwrXry44uLiblp++fJlXbt2zabcy8tLxYoVM/YHAAAAbhc5SqKXLl1aL730kl566SXt2bNHX3zxhT788EO99957evjhhxUWFqYWLVrIyytHbw8AAADgJlwdkyclJcnHx8dmm4+Pj7GEzI3Kk5OTjdf29neUyeRsy2+NzfEcObbpuj9dVD+v+8GVTHfgObkb+tj16GPXo49djz52PfrY9fKqb285y33//ffr/vvvV8eOHTV16lRt3bpVW7duVYkSJdSzZ0/17t1bnp6eudFWAAAAANlwRUzu6+urxMREm21ms1kFChQwyq9PiJvNZhUpUkS+vr7G6+vL/fz8HG6Dj0/eX0d4e3tKJsnDwSsyD5NJJg/X1vfw9kxv1x3CZJI8PT1lMkn/ew4tchl97Hr0sevRx65HH7sefex6GX3sareURI+NjdW6deu0bt06RUdHq0KFChoyZIiaNWumrVu3KiIiQkePHtWUKVNyq70AAAAAMnFVTB4QEKCjR4/abEtISDCWaAkICFBCQkKW8vvuu0/FihWTr6+vEhISVLVqVUlSamqqEhMTnVpqxmy25PnMrZQUi7ysUpqDV7ppVqusaa6tb0mxKCXF4lD920FGIiE11UJCwUXoY9ejj12PPnY9+tj16GPXy6sfKHKURF+5cqXWrVunPXv2yNfXV61bt9akSZNsHhxUo0YNXbx4UZ9//jlJdAAAACCXuTomDw4O1vz585WcnGzMPo+MjFT9+vWN8sjISKN+UlKSDh48qEGDBsnDw0NBQUGKjIxUo0aNJEn79u2Tl5eXatWq5VQ78vqC0+Z4jhzbet2fLqp/J154W6135nm5E/rY9ehj16OPXY8+dj36+PaXoyT6mDFjFBwcrDfffFNPPvmkChUqlG29mjVrqnPnzrfUQAAAAABZuTomb9iwocqUKaPw8HANGDBAW7Zs0f79+zV58mRJUvv27bVgwQLNnz9fzZs3V0REhMqVK2ckzbt166axY8eqRo0aKlWqlN5880116tTJqeVcAAAAAHeQoyT6hg0bVK1aNVksFmPNmeTkZKWkpKhw4cJGvXbt2jn8nmazWWFhYRozZowReE+cOFGffPKJTb0xY8bo+eefN9oxY8YMxcfH6+GHH9aECRPk7+8vSbJarZo+fbpWrVqltLQ0dejQQcOGDZOHh0dOThkAAABwK66IyTPz9PTUBx98oFGjRiksLEwVK1ZURESEAgMDJUnlypXT+++/r7feeksREREKDQ1VRESETP9bf+Wpp55SbGysxo4dK7PZrMcff1zDhw+/tZMGAAAA8kGOkuiVKlXSG2+8od9//12rV6+WJO3Zs0d9+/ZV9+7dNXz4cKeS1deuXdPQoUN15MgRm+3Hjh3T0KFD9dxzzxnbMmbY7N+/X6NGjdK4ceNUq1YtTZo0SeHh4Zo3b54kaeHChdqwYYNmz56t1NRUDR8+XMWLF1efPn1ycsoAAACAW8ntmFySDh8+bPO6YsWKWrp0qd36jz76qB599FG75X379lXfvn2dagMAAADgbnKURJ81a5a+/PJLDR482NhWu3ZtDRs2TO+//77uvfdeh4Plo0ePaujQobJmszDQsWPH1KdPn2wfPrR06VK1adPGmFkzdepUNW/eXDExMSpfvryWLFmiwYMHG2tCDhs2TDNnzrzjkugWS6qio087tU+FCpXk5XVLz5QFAABAPsvNmBwAAACAfTnKpK5fv14jRoxQly5djG3FihVTr1695OXlpSVLljgcsO/evVuNGjXSq6++qpCQEGP7lStXdPbsWVWqVCnb/aKiovTiiy8ar8uUKaPAwEBFRUXJx8dHZ86c0QMPPGCU169fX7GxsTp37pxKlSrl3Am7sQtxpxWXYlZ0oVSH6iecOaV2kqpUqebSdgEAAMC1cjMmBwAAAGBfjpLoFy9eVPny5bMtq1KliuLi4hx+r27dumW7/dixYzKZTJo7d662b9+uYsWK6d///rextEt2yfDixYsrLi5O8fHxkmRTXqJECUlSXFzcHZVElyT/0mVVulLV/G4GAAAA8lBuxuQAAAAA7MtREr1KlSratGmTmjRpkqXshx9+UMWKFW+5YX/99ZdMJpOqVKmi559/Xr/88ovGjBmjQoUKqVWrVkpOTpaPj4/NPj4+PjKbzUpOTjZeZy6T0h9g6oz/PRcpz2QczyQp6wI3uX8cOM4YG/rO7TA27ouxcV+MjftibNyTO45LXsTkAAAAAHKYRO/Ro4dGjhypxMREtWzZUsWLF9eFCxe0ZcsWffPNN5o8efItN6xdu3Zq3ry5ihUrJkmqVauWTpw4oWXLlqlVq1by9fXNkhA3m83y8/OzSZj7+voaf5ckPz8/h9vg4+N5y+fhLG9vT5lMFpk80hPpN2PykEwmkxx9ZpTJI/0Y3t55f263O5NJ8vT0lMkkZbOEP/IRY+O+GBv3xdi4L8bGPbnjuORFTA4AAAAgh0n0du3a6Z9//tEHH3ygb7/91th+7733asyYMcbDPm+FyWQyEugZqlSpop07d0qSAgIClJCQYFOekJCgkiVLKiAgQJIUHx+vcuXKGX+XlO1DSu0xmy15PtsoJcUiq9Uqa5pjM9GtaZLValVammPvb01LP0ZKiuWW2nk3yrhoTk21uM3FM9IxNu6LsXFfjI37YmzckzuOS17E5AAAAABymESXpH/961/q1q2bjh8/rsTERBUpUkRVqlSRh6NTom9i5syZ2rt3rxYtWmRsO3TokKpUqSJJCg4OVmRkpMLCwiRJZ86c0ZkzZxQcHKyAgAAFBgYqMjLSSKJHRkYqMDDQ6fXQ8/oiKeN4rj6su1z83Y6sVvrPXTE27ouxcV+MjftibNyTu42Lq2NyAAAAALeQRJdkrFnuCs2bN9f8+fO1YMECtWrVSjt27NAXX3yhJUuWSJK6du2q7t27KyQkREFBQZo0aZKaNWtmPFypa9eueuedd1S6dGlJ0vTp09W7d2+XtBUAAADIL66MyQEAAADkMIl+4cIFTZo0SVu3blVSUpKs103HMZlMOnjw4C01rF69epo5c6ZmzZqlmTNnqmzZspo+fbpCQ0MlSaGhoRo/frxmzZqlS5cuqUmTJpowYYKxf58+fXT+/HkNGjRInp6e6tChg3r16nVLbQIAAADcRV7E5AAAAABymEQfP368tmzZoqeeekqlS5fOtdtFDx8+bPO6ZcuWatmypd36YWFhxnIu1/P09FR4eLjCw8NzpW0AAACAO3FVTA4AAADAVo6S6Nu3b9frr7+uzp0753Z7AAAAADiAmBwAAADIGzmaruLt7W2sPQ4AAAAg7xGTAwAAAHkjR0n0Vq1aacOGDbndFgAAAAAOIiYHAAAA8kaOlnOpXbu2ZsyYoZiYGAUHB6tAgQI25SaTSQMHDsyVBgIAAADIipgcAAAAyBs5frCoJP3yyy/65ZdfspQTsAMAAACuRUwOAAAA5I0cJdEPHTqU2+0AAAAA4ARicgAAACBv5GhN9Mz+/vtvHTt2TGazWRaLJTfaBAAAAMAJxOQAAACA6+Q4ib5r1y517NhRDRs21NNPP60jR45o6NChevvtt3OzfQAAAADsICYHAAAAXC9HSfSff/5Zffr0UYECBTRs2DBZrVZJUq1atbRkyRItXLgwVxsJAAAAwBYxOQAAAJA3cpREnzFjhh577DF98skn6tmzpxGw9+/fXy+88IJWrlyZq40EAAAAYIuYHAAAAMgbOUqi//HHH2rfvr0kyWQy2ZQ1adJEsbGxt94yAAAAAHYRkwMAAAB5I0dJ9MKFCys+Pj7bsjNnzqhw4cK31CgAAAAAN0ZMDgAAAOSNHCXRH3vsMb333nv67bffjG0mk0lxcXGaO3eumjVrllvtAwAAAJANYnIAAAAgb3jlZKehQ4cqKipKnTp1UokSJSRJQ4YMUVxcnMqUKaMhQ4bkaiMBAAAA2CImBwAAAPJGjpLoRYsW1cqVK/XFF19o586dSkxMVOHChdW9e3eFhYXJz88vt9sJAAAAIBNi8rtDqsWimOhop/apUKGSvLxydKkHAACAbOQ4svLx8VGnTp3UqVOn3GwPAAAAAAcRk9/5Tl+8oLQft8gzsKxD9U9dOC+176wqVaq5uGUAAAB3jxwl0b/44oub1mnXrl1O3hoAAACAA4jJ7x6B/v6qWrq0w/UtLmwLAADA3ShHSfSRI0dmu91kMsnT01Oenp4E7AAAAIALEZMDAAAAeSNHSfTvv/8+y7arV6/q119/1YcffqiIiIhbbhgAAAAA+4jJAQAAgLyRoyR62bLZr8dXvXp1paSkaMKECfrss89uqWEAAAAA7CMmBwAAAPKGR26/Yc2aNXXgwIHcflsAAAAADiImBwAAAHJPribRzWazVq1apeLFi+fm2wIAAABwEDE5AAAAkLtytJxLixYtZDKZbLalpaXp4sWLunbtmkaMGJErjQMAAACQPWJyAAAAIG/kKInesGHDLAG7JBUqVEjNmzfXQw89dMsNAwAAAGAfMTkAAACQN3KURH/77bdzux0AAAAAnEBMDgAAAOSNHCXRT58+7VT9wMDAnBwGAAAAgB3E5AAAAEDeyLU10W/kjz/+yMlhAAAAANhBTA4AAADkjRwl0WfMmKE33nhDderU0TPPPKOAgABdvHhRP/zwg7755hu99NJLKlu2bG63FQAAAMD/EJMDAAAAeSNHSfR169apefPmWdZhfPLJJ1W8eHHt2bNHgwYNypUGAgAAAMiKmBwAAADIGx452ennn39W27Ztsy175JFHFBkZeUuNAgAAAHBjxOQAAABA3shREv3ee+9VVFRUtmU///yzAgICbqlRAAAAAG6MmBwAAADIGzlazqVDhw6aM2eOkpKS1KJFC/n7+yshIUEbN27UsmXLNGbMmNxuJwAAAIBMiMkBAACAvJGjJPqAAQP0999/a9GiRVqwYIEkyWq1ys/PT6+++qq6dOmSq40EAAAAYIuYHAAAAMgbOUqim0wmjRw5UgMGDNC+fft06dIl3XvvvQoJCVGhQoVyu40AAAAArkNMDgAAAOSNHCXRMxQqVEilSpWSJIWEhCg1NTVXGgUAAADAMcTkAAAAgGvl6MGikrRu3To1a9ZMzz33nPr376+TJ09q5MiRevnll2U2m3OzjQAAAACykRcx+ZkzZ9SvXz/df//9atGihRYtWmSUHTx4UB07dlRwcLDat2+v33//3WbfDRs2qGXLlgoODtbAgQN14cKFXGkTAAAAkJdylET/+uuvNWLECDVu3Fjvvvuu0tLSJEmtWrXStm3b9MEHH+RqIwEAAADYyquY/JVXXlHBggW1Zs0avf7665oxY4a+++47Xb16VX379lWDBg20Zs0ahYaGql+/frp69aokaf/+/Ro1apQGDRqk5cuX6/LlywoPD8+VNgEAAAB5KUfLucydO1ddunTRm2++KYvFYmxv3769Lly4oBUrVuiVV17JrTYCAAAAuE5exOSXLl3Svn37NGHCBFWqVEmVKlVS06ZN9fPPP+vSpUvy9fXVa6+9JpPJpFGjRmn79u3auHGjwsLCtHTpUrVp00bt2rWTJE2dOlXNmzdXTEyMypcvf0vtAgAAAPJSjmaiHz9+XK1atcq2LDg4WGfPnr2lRgEAAAC4sbyIyQsUKCA/Pz+tWbNGKSkp+uuvv7Rnzx7dd999ioqKUv369WUymSSlP+j0/vvv1759+yRJUVFRatCggfFeZcqUUWBgoKKiom65XQAAAEBeylESvXjx4jp27Fi2ZceOHVPx4sVvqVEAAAAAbiwvYnJfX1+NHTtWy5cvV3BwsNq0aaNHHnlEHTt2VHx8vPFA08xtiouLkySdO3fuhuUAAADA7SJHy7k8+eSTmjVrlkqVKqVHH31UUvrMk99//10ffPCB2rZtm6uNBAAAAGArr2LyY8eOqXnz5vr3v/+tI0eOaMKECXrwwQeVlJQkHx8fm7o+Pj7GA02Tk5NvWO6o/010zzM2x3Pk2Kbr/nST+nndb84w3QZtvN3Rx65HH7sefex69LHr0ceul1d9m6Mk+iuvvKI///xTr7zyijw80iezd+/eXVevXlWDBg30n//8J1cbCQAAAMBWXsTkP//8s1atWqVt27apQIECCgoK0tmzZzVnzhyVL18+S0LcbDarQIECktJnsWdX7ufn5/DxfXw8b/kcnOXt7SmZJA8Hr8g8TCaZPNyrvoe3Z/p5uCmTSfL09JTJJFmt+d2aOxN97Hr0sevRx65HH7sefex6GX3sajlKovv4+Oijjz7Sf//7X+3cuVOJiYkqXLiwGjZsqEcffdRYFxEAAACAa+RFTP7777+rYsWKRmJckmrXrq25c+eqQYMGSkhIsKmfkJBgLOESEBCQbXnJkiUdPr7ZbMnzmVspKRZ5WaU0B69006xWWdPcq74lxaKUFMvNK+eTjERCaqqFhIKL0MeuRx+7Hn3sevSx69HHrpdXP1DkKInep08fvfDCC2rSpImaNGmS220CAAAAcBN5EZOXKlVKJ0+elNlsNpZm+euvv1SuXDkFBwfrww8/lNVqlclkktVq1Z49e9S/f39J6Q83jYyMVFhYmCTpzJkzOnPmjIKDg51qQ15fcNocz5FjW6/7M5/rp6ZaFBMd7eCbp6tQoZK8vHJ0aXhLrFZm5bkafex69LHr0ceuRx+7Hn18+8tRpLRnzx5mmwMAAAD5KC9i8hYtWmjatGkaPXq0XnrpJR0/flxz587Vq6++qtatW2v69OmaNGmSunTpos8//1xJSUlq06aNJKlr167q3r27QkJCFBQUpEmTJqlZs2YqX768S9t8tzt98YLSftwiz8CyDtU/deG81L6zqlSp5uKWAQAA3L5ylERv2rSpvvzyS9WvX1/e3t653SYAAAAAN5EXMXnhwoW1aNEiTZo0SR06dJC/v79eeuklde7cWSaTSfPmzdMbb7yhFStWqGbNmpo/f74KFiwoSQoNDdX48eM1a9YsXbp0SU2aNNGECRNc0k7YCvT3V9XSpR2u774LvwAAALiHHCXRfX199eWXX+qbb75R1apVjUA5g8lk0uLFi3OlgQAAAACyyquYvFq1alq4cGG2ZfXq1dPatWvt7hsWFmYs5wIAAADcrnKURI+Li1NoaKjx2nrdoj7XvwYAAACQu4jJAQAAgLzhcBL922+/VePGjVWkSBF98sknrmwTAAAAgGwQkwMAAAB5z8PRiv/5z3904sQJm20ffvihzp8/n9ttAgAAAJANYnIAAAAg7zmcRL/+dlCLxaJ3331XcXFxud4oAAAAAFkRkwMAAAB5z+EkenZYZxEAAADIX8TkAAAAgGvdUhIdAAAAAAAAAIA7GUl0AAAAAAAAAADsuOUkuslkyo12AAAAAMghYnIAAADAdbycqTxw4ED5+PjYbOvfv7+8vb1ttplMJm3evPnWWwcAAADABjE5AAAAkLccTqI/99xzrmwHAAAAgJsgJgcAAADynsNJ9MmTJ7uyHQAAAABugpgcAAAAyHs8WBQAAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANjhld8NAAAAAADkj1SLRTHR0Q7Xr1Chkry8uIwEAAB3F6IfAAAAALhLnb54QWk/bpFnYNmb1j114bzUvrOqVKmWBy0DAABwHyTRAQAAAOAuFujvr6qlSztU1+LitgAAALgj1kQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADvcJoluNpvVtm1b7dq1y9gWExOjXr16KSQkRE8++aR27Nhhs89PP/2ktm3bKjg4WD169FBMTIxN+aJFi9S0aVOFhobq9ddfV1JSUp6cCwAAAAAAAADgzuAWSfRr165pyJAhOnLkiLHNarVq4MCBKlGihFavXq1nn31WgwYN0unTpyVJp0+f1sCBAxUWFqZVq1bJ399fAwYMkNVqlSRt2rRJs2fP1vjx47V48WJFRUVp2rRp+XJ+AAAAAAAAAIDbU74n0Y8ePapOnTopOjraZvvOnTsVExOj8ePHq2rVqurXr59CQkK0evVqSdLKlStVt25d9e7dW9WrV9fkyZMVGxur3bt3S5KWLFminj17qnnz5qpXr57GjRun1atXMxsdAAAAAAAAAOCwfE+i7969W40aNdLy5ctttkdFRal27doqWLCgsa1+/frat2+fUd6gQQOjzM/PT3Xq1NG+fftksVj022+/2ZSHhIQoJSVFhw4dcu0JAQAAAAAAAADuGF753YBu3bpluz0+Pl6lSpWy2Va8eHHFxcXdtPzy5cu6du2aTbmXl5eKFStm7O8ok8mp6rcs43gmSdY8OA4cZ4wNfed2GBv3xdi4L8bGfTE27olxAQAAAO5e+Z5EtycpKUk+Pj4223x8fGQ2m29anpycbLy2t78jfHw8c9L0W+Lt7SmTySKTR3oi/WZMHpLJZJKHg/cUmDzSj+HtnffndrszmSRPT0+ZTJLVlb9wwGmMjftibNwXY+O+GBv3xLgAAAAAdy+3TaL7+voqMTHRZpvZbFaBAgWM8usT4mazWUWKFJGvr6/x+vpyPz8/h9tgNlvyfLZRSopFVqtV1jTHZqJb09IfwpqW5tj7W9PSj5GSYrmldt6NMi6aU1MtXDy7GcbGfTE27ouxcV+MjXtiXAAAAIC7l9sm0QMCAnT06FGbbQkJCcYSLQEBAUpISMhSft9996lYsWLy9fVVQkKCqlatKklKTU1VYmKiSpYs6VQ78voiKeN4rj4sF385Z7XSf+6KsXFfjI37YmzcF2PjnhgXAAAA4O6T7w8WtSc4OFgHDhwwlmaRpMjISAUHBxvlkZGRRllSUpIOHjyo4OBgeXh4KCgoyKZ837598vLyUq1atfLuJAAAAAAAAAAAtzW3TaI3bNhQZcqUUXh4uI4cOaL58+dr//796tChgySpffv22rNnj+bPn68jR44oPDxc5cqVU6NGjSSlP7B0wYIF2rx5s/bv368333xTnTp1cmo5FwAAAAAAAADA3c1tk+ienp764IMPFB8fr7CwMH355ZeKiIhQYGCgJKlcuXJ6//33tXr1anXo0EGJiYmKiIiQ6X+LmD/11FPq16+fxo4dq969e6tevXoaPnx4fp4SAAAAAAAAAOA241Zroh8+fNjmdcWKFbV06VK79R999FE9+uijdsv79u2rvn375lr7AAAAAOBulWqxKCY62ql9KlSoJC8vt7rsBAAAcBrRDAAAAAC7zGazJk+erA0bNsjb21sdOnTQq6++KpPJpIMHD+qNN97Qn3/+qWrVqmncuHGqW7euse+GDRs0Y8YMxcfH6+GHH9aECRPk7++fj2eDW3H64gWl/bhFnoFlHap/6sJ5qX1nValSzcUtAwAAcC23Xc4FAAAAQP6bOHGifvrpJy1YsEDTp0/XihUrtHz5cl29elV9+/ZVgwYNtGbNGoWGhqpfv366evWqJGn//v0aNWqUBg0apOXLl+vy5csKDw/P57PBrQr091fV0qUd+q+cf/H8bi4AAECuYCY6AAAAgGwlJiZq9erVWrhwoerVqydJ6t27t6KiouTl5SVfX1+99tprMplMGjVqlLZv366NGzcqLCxMS5cuVZs2bdSuXTtJ0tSpU9W8eXPFxMSofPny+XhWAAAAgHOYiQ4AAAAgW5GRkSpUqJAaNmxobOvbt68mT56sqKgo1a9fXyaTSZJkMpl0//33a9++fZKkqKgoNWjQwNivTJkyCgwMVFRUVJ6eAwAAAHCrSKIDAAAAyFZMTIzKli2rL774Qq1bt9Zjjz2miIgIpaWlKT4+XqVKlbKpX7x4ccXFxUmSzp07d8NyAAAA4HbBci4AAAAAsnX16lWdPHlSn3/+uSZPnqz4+HiNHTtWfn5+SkpKko+Pj019Hx8fmc1mSVJycvINyx31v4nuecbmeI4c23Tdn3dy/Ry+9/VjaLKzHbmHPnY9+tj16GPXo49djz52vbzqW5LoAAAAALLl5eWlK1euaPr06Spbtqwk6fTp01q2bJkqVqyYJSFuNptVoEABSZKvr2+25X5+fg4f38fH8xbPwHne3p6SSfJw8IrMw2SSyePuqJ+T9/bw9kzv00xMJsnT01Mmk2S1OvRWcBJ97Hr0sevRx65HH7sefex6GX3saiTRAQAAAGSrZMmS8vX1NRLoklS5cmWdOXNGDRs2VEJCgk39hIQEYwmXgICAbMtLlizp8PHNZkuez9xKSbHIyyqlOXilm2a1ypp2d9TPyXtbUixKSbHYbM9IJKSmWkgouAh97Hr0sevRx65HH7sefex6efUDBUl0AAAAANkKDg7WtWvXdPz4cVWuXFmS9Ndff6ls2bIKDg7Whx9+KKvVKpPJJKvVqj179qh///7GvpGRkQoLC5MknTlzRmfOnFFwcLBTbcjrC06b4zlybOt1f97J9XP43vbG0GplVp6r0ceuRx+7Hn3sevSx69HHtz8eLAoAAAAgW1WqVFGzZs0UHh6uQ4cO6ccff9T8+fPVtWtXtW7dWpcvX9akSZN09OhRTZo0SUlJSWrTpo0kqWvXrlq3bp1WrlypQ4cO6bXXXlOzZs1Uvnz5fD4rAAAAwDkk0QEAAADY9c4776hChQrq2rWrRowYoX/961/q3r27ChUqpHnz5hmzzaOiojR//nwVLFhQkhQaGqrx48crIiJCXbt2VdGiRTV58uR8PhsAAADAeSzncpexWFIVHX3aqX0qVKgkLy8+KgAAAHejwoULa+rUqdmW1atXT2vXrrW7b1hYmLGcCwAAAHC7IjN6l7kQd1pxKWZFF0p1qH7CmVNqJ6lKlWoubRcAAAAAAAAAuCOS6Hch/9JlVbpS1fxuBgAAAAAAAAC4PdZEBwAAAAAAAADADpLoAAAAAAAAAADYQRIdAAAAAAAAAAA7SKIDAAAAAAAAAGAHSXQAAAAAAAAAAOwgiQ4AAAAAAAAAgB0k0QEAAAAAAAAAsIMkOgAAAAAAAAAAdpBEBwAAAAAAAADADpLoAAAAAAAAAADYQRIdAAAAAAAAAAA7SKIDAAAAAAAAAGCHV343AHev1NRURUefcGqfChUqycuLjy0AAAAAAACAvEE2ErnKmcR4dHS09vxtUYky5R2qn3DmlNpJqlKlWo7bBwAAAAAAAADOIImOXBUdfUJfHDihEmXK3bTun0dPqXytIJWuVDUPWgYAAAAAAAAAziOJjlxXokw5hxLj8bHRedAaAAAAAAAAAMg5kugAAAAAgFyXarEoJjr7iTPe3p5KSbFk2c4zkAAAgDsiOsENWSypio4+7XD96OhoWQuVdmGLAAAAANwOTl+8oLQft8gzsKxtgUmSySRPq1Wy/v/mUxfOS+078wwkAADgdkii44YuxJ1WXIpZ0YVSHaqfvs75vS5pi7MJfYmZLAAAAEB+CvT3V9XS102yMUkeJpPSrkuiS1LWuekAAAD5j+wibsq/dFmHH/7pynXOnU3oJ5w5pXYSM1kAAAAAAAAA5BhJdNxWnEnoAwAAAAAAAMCt8sjvBgAAAAAAAAAA4K5IogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOkugAAAAAAAAAANhBEh0AAAAAAAAAADtIogMAAAAAAAAAYAdJdAAAAAAAAAAA7CCJDgAAAAAAAACAHSTRAQAAAAAAAACwgyQ6AAAAAAAAAAB2kEQHAAAAAAAAAMAOr/xuAAAAAAAAqRaLYqKjndqnQoVK8vLishYAALgW0QYAAAAAIN+dvnhBaT9ukWdgWYfqn7pwXmrfWVWqVHNxywAAwN2OJDoAAAAAwC0E+vuraunSDte3uLAtAAAAGVgTHQAAAAAAAAAAO0iiAwAAAAAAAABgB0l0AAAAAAAAAADsIIkOAAAAAAAAAIAdJNEBAAAAOKRv374aOXKk8frgwYPq2LGjgoOD1b59e/3+++829Tds2KCWLVsqODhYAwcO1IULF/K6yQAAAMAtI4kOAAAA4Ka++uorbdu2zXh99epV9e3bVw0aNNCaNWsUGhqqfv366erVq5Kk/fv3a9SoURo0aJCWL1+uy5cvKzw8PL+aDwAAAOQYSXQAAAAAN5SYmKipU6cqKCjI2Pb111/L19dXr732mqpWrapRo0bpnnvu0caNGyVJS5cuVZs2bdSuXTvVqlVLU6dO1bZt2xQTE5NfpwEAAADkCEl0AAAAADc0ZcoUPfvss6pWrZqxLSoqSvXr15fJZJIkmUwm3X///dq3b59R3qBBA6N+mTJlFBgYqKioqDxtOwAAAHCrSKIDAAAAsOvnn3/Wr7/+qgEDBthsj4+PV6lSpWy2FS9eXHFxcZKkc+fO3bAcAAAAuF145XcDAAAAALina9eu6Y033tDYsWNVoEABm7KkpCT5+PjYbPPx8ZHZbJYkJScn37DcUf+b6J5nbI7nyLFN1/15J9d3xXtn199Ovn9ef0ZuByb6xuXoY9ejj12PPnY9+tj18qpvSaIDAAAAyNbs2bNVt25dNW3aNEuZr69vloS42Ww2ku32yv38/Bw+vo+PZw5afWu8vT0lk+Th4BWZh8kkk8fdUT8339tkkjyuy5bn5P09vD3Txww2TCbJ09NTJpNkteZ3a+5M9LHr0ceuRx+7Hn3sehl97Gok0QEAAABk66uvvlJCQoJCQ0MlyUiKb9q0SW3btlVCQoJN/YSEBGMJl4CAgGzLS5Ys6fDxzWZLns/cSkmxyMsqpTl4pZtmtcqadnfUz8339pApy/acvL8lxaKUFItD9e8mGcma1FQLSRsXoY9djz52PfrY9ehj18urHyhIogMAAADI1ieffKLU1FTj9TvvvCNJGjZsmH755Rd9+OGHslqtMplMslqt2rNnj/r37y9JCg4OVmRkpMLCwiRJZ86c0ZkzZxQcHOxUG/L6gtPmeI4c23rdn3dy/dx6b1M2dW7h/UlK2Ge10j+uRh+7Hn3sevSx69HHtz+S6AAAAACyVbZsWZvX99xzjySpYsWKKl68uKZPn65JkyapS5cu+vzzz5WUlKQ2bdpIkrp27aru3bsrJCREQUFBmjRpkpo1a6by5cvn+XkAAAAAt8IjvxsAAAAA4PZTqFAhzZs3z5htHhUVpfnz56tgwYKSpNDQUI0fP14RERHq2rWrihYtqsmTJ+dzqwEAAADnMRMdAAAAgEPefvttm9f16tXT2rVr7dYPCwszlnMBAAAAblfMRAcAAAAAAAAAwA6S6AAAAAAAAAAA2EESHQAAAAAAAAAAO0iiAwAAAAAAAABgBw8WBQAAAADcdlItFsVERzu1T4UKleTlxWUwAABwDtEDAAAAAOC2c/riBaX9uEWegWUdqn/qwnmpfWdVqVLNxS0DAAB3GpLoAAAAAIDbUqC/v6qWLu1wfYsL2wIAAO5crIkOAAAAAAAAAIAdJNEBAAAAAAAAALCDJDoAAAAAAAAAAHaQRAcAAAAAAAAAwA63T6J/9913qlmzps1/gwcPliQdPHhQHTt2VHBwsNq3b6/ff//dZt8NGzaoZcuWCg4O1sCBA3XhwoX8OAUAAAAAAAAAwG3KK78bcDNHjx5V8+bNNWHCBGObr6+vrl69qr59++rpp5/W22+/rWXLlqlfv3767rvvVLBgQe3fv1+jRo3SuHHjVKtWLU2aNEnh4eGaN29ePp4N8pLFkqro6NNO7VOhQiV5ebn9PwsAAAAAAAAAecTts4XHjh1TjRo1VLJkSZvtq1atkq+vr1577TWZTCaNGjVK27dv18aNGxUWFqalS5eqTZs2ateunSRp6tSpat68uWJiYlS+fPl8OBPktQtxpxWXYlZ0oVSH6iecOaV2kqpUqebSdgEAAAAAAAC4fdwWSfSHHnooy/aoqCjVr19fJpNJkmQymXT//fdr3759CgsLU1RUlF588UWjfpkyZRQYGKioqCiS6HcR/9JlVbpS1fxuBgAAAAAAAIDblFsn0a1Wq44fP64dO3Zo3rx5slgsat26tQYPHqz4+HhVq2Y7Y7h48eI6cuSIJOncuXMqVapUlvK4uDin2vC/HH2eyTieSZI1bw+N/7E35sbY5PFnAjfH2LgvxsZ9MTbui7FxT4wLAAAAcPdy6yT66dOnlZSUJB8fH82YMUOnTp3SxIkTlZycbGzPzMfHR2azWZKUnJx8w3JH+Ph43vpJOMnb21Mmk0Umj/RE+s2YPNJn4Xs4+IhYd6rvTm3JqO/t7Slv7+zH3WSSPD09ZTJJVn7hcCuMjftibNwXY+O+GBv3xLgAAAAAdy+3TqKXLVtWu3btUtGiRWUymXTfffcpLS1Nw4cPV8OGDbMkxM1mswoUKCAp/eGj2ZX7+fk5fHyz2ZLns41SUiyyWq2ypjk2E92alj5jPy3Nsfd3p/ru1JaM+ikpFqWkWLItz7hoTk21cPHsZhgb98XYuC/Gxn0xNu6JcQEAAADuXm6dRJekYsWK2byuWrWqrl27ppIlSyohIcGmLCEhwVjCJSAgINvy6x9QejN5fZGUcTyuzfLPzcbcamUGmrtibNwXY+O+GBv3xdi4J8YFAAAAuPs4uNBF/vjxxx/VqFEjJSUlGdv++OMPFStWTPXr19fevXtl/d9VjNVq1Z49exQcHCxJCg4OVmRkpLHfmTNndObMGaMcAAAAAAAAAICbceskemhoqHx9fTV69Gj99ddf2rZtm6ZOnaoXXnhBrVu31uXLlzVp0iQdPXpUkyZNUlJSktq0aSNJ6tq1q9atW6eVK1fq0KFDeu2119SsWTOVL18+n88KAAAAAAAAAHC7cOskeqFChbRgwQJduHBB7du316hRo9S5c2e98MILKlSokObNm6fIyEiFhYUpKipK8+fPV8GCBSWlJ+DHjx+viIgIde3aVUWLFtXkyZPz+YwAAAAAAAAAALcTt18TvXr16lq4cGG2ZfXq1dPatWvt7hsWFqawsDBXNQ0AAAAAAAAAcIdz65noAAAAAAAAAADkJ5LoAAAAAAAAAADYQRIdAAAAAAAA+L/27j2uqirv4/j3CAqYY4YYCk456gMiIiCGliBppj7ectQmKyuzGZ2k7Oa1GdPG0tK85K2wNK1Mm7LU7PJkk2VmWkNJY+Z4Kw+IFxBJTe5nPX847vEIRw/K4SB83q8XLz17r7332mttzv6tH/usAwAukEQHAAAAAAAAAMAFkugAAAAAAAAAALhAEh0AAAAAAAAAABdIogMAAAAAAAAA4AJJdAAAAAAAAAAAXCCJDgAAAAAAAACACyTRAQAAAAAAAABwwdfbFQCqipKSYtntmectU7u2j4qKSqzX11zTTL6+/BoBAAAAVV1xSYnS7fZybUO8DwAAJJLogCXnUKYOFRXKXq/YZRlbrWIZx+n/Zx/MUH9JzZu3rJT6AQAAALh4mcdy5Phig3xCQt0qn5FzVBp4G/E+AAAgiQ6cLbBxqBo3a+Fyfa1aksNRiRUCAAAAUGFCAgPVonFjt8uXXLgIAACoAZgTHQAAAAAAAAAAF0iiAwAAAAAAAADgAkl0AAAAAAAAAABcIIkOAAAAAAAAAIALJNEBAAAAAAAAAHDB19sVAAAAAACgqikuKVG63V6uba65ppl8fRlmAwBQ3XB3BwAAAADgHJnHcuT4YoN8QkLdKp+Rc1QaeJuaN2/p4ZoBAIDKRhIdAAAAAIAyhAQGqkXjxm6XL/FgXQAAgPcwJzoAAAAAAAAAAC6QRAcAAADg0uHDhzVq1CjFx8crMTFR06ZNU0FBgSQpPT1dQ4cOVUxMjHr16qVNmzY5bbt582b16dNH0dHRuvvuu5Wenu6NUwAAAAAuCUl0AAAAAGUyxmjUqFHKy8vT8uXLNXv2bG3YsEFz5syRMUbJyckKCgrSqlWrdMstt+iBBx5QZmamJCkzM1PJyckaMGCA3n77bQUGBmrkyJEyxnj5rAAAAIDyYU50AAAAAGXat2+ftm3bpi+//FJBQUGSpFGjRunZZ59V586dlZ6erpUrV6pu3bpq0aKFvvrqK61atUoPPvig3nrrLbVp00bDhg2TJE2bNk2dOnXS119/rQ4dOnjztAAAAIBy4Ul0AAAAAGVq1KiRXn75ZSuBfsbJkyeVlpam1q1bq27dutbyuLg4bdu2TZKUlpam9u3bW+sCAgIUGRlprQcAAAAuFyTRAQAAAJSpfv36SkxMtF47HA69/vrr6tixo7KysnT11Vc7lW/YsKEOHTokSRdcDwAAAFwumM4FAAAAgFtmzJihHTt26O2339bSpUtVp04dp/V16tRRYWGhJCkvL++8691ls11ancvL6XjuHNt2zr/Vubwn9l1We1eFc72E8pV9zZbFVoXqUl3Rxp5HG3sebex5tLHnVVbbkkQHAAAAcEEzZszQsmXLNHv2bIWFhcnPz0+5ublOZQoLC+Xv7y9J8vPzK5UwLywsVP369d0+Zp06Ppdc7/KqXdtHskm13ByR1bLZZKtVM8pX5L5tNqnWOdnpqnSuF1Pe4XDo4MGM09eQm669tpl8fSt+WG6zST4+PrLZJL7L1zNoY8+jjT2PNvY82tjzzrSxp5FEBwAAAHBeU6ZM0YoVKzRjxgz16NFDkhQcHKw9e/Y4lcvOzramcAkODlZ2dnap9REREW4ft7CwpNKf3CoqKpGvkRxujnQdxsg4akb5itx3LdlKLa9K53ox5TNyjqro03/IERrqfvkBt6lFi5ZulS+PM8ma4uISkjYeQht7Hm3sebSx59HGnldZf6AgiQ4AAADApfnz52vlypWaNWuWevbsaS2Pjo7WokWLlJ+fbz19npqaqri4OGt9amqqVT4vL087duzQAw88UK7jV/aA0+l47hzbnPNvdS5fUfu2lVGmIvfvxfIhgYFqEdzY7fIl8uw1bgxPPnoabex5tLHn0caeRxtf/vhiUQAAAABl2rt3rxYuXKg//elPiouLU1ZWlvUTHx+vJk2aaMKECdq9e7cWLVqk77//XoMGDZIkDRw4UN9++60WLVqk3bt3a8KECWratKk6dOjg5bMCAAAAyockOgAAAIAy/eMf/1BJSYleeOEFJSQkOP34+Pho4cKFysrK0oABA7R27VotWLBAISEhkqSmTZtq3rx5WrVqlQYNGqTc3FwtWLBANr5ZCwAAAJcZpnMBAAAAUKbhw4dr+PDhLtdfe+21ev31112uT0pKUlJSkieqBgAAAFQankQHAAAAAAAAAMAFnkQHLlJJSbHs9sxybXPNNc3k68uvHQAAAAAAAHC5IJsHXKScQ5k6VFQoe71it8pnH8xQf0nNm7f0aL0AAAAAAAAAVByS6MAlCGwcqsbNWni7GgAAAAAAAAA8hDnRAQAAAAAAAABwgSQ6AAAAAAAAAAAukEQHAAAAAAAAAMAFkugAAAAAAAAAALhAEh0AAAAAAAAAABdIogMAAAAAAAAA4IKvtysAAAAAAEBNU1xSonS7vVzbXHNNM/n6MowHAKCycfcFAAAAAKCSZR7LkeOLDfIJCXWrfEbOUWngbWrevKWHawYAAM5FEh0AAAAAAC8ICQxUi8aN3S5f4sG6AAAA15gTHQAAAAAAAAAAF0iiAwAAAAAAAADgAkl0AAAAAAAAAABcYE50oJKUlBTLbs8s1zbXXNNMvr78mgIAAAAAAADeQnYOqCQ5hzJ1qKhQ9nrFbpXPPpih/pKaN2/p0XoBAAAAAAAAcI0kOlCJAhuHqnGzFt6uBgAAAAAAAAA3MSc6AAAAAAAAAAAukEQHAAAAAAAAAMAFkugAAAAAAAAAALjAnOgAAAAAAFRxxSUlSrfb3S7fsmULSTbPVQgAgBqEJDoAAAAAAFVc5rEcOb7YIJ+Q0AuW3X80Swc7JapJk6Zu7/+aa5rJ15cUAQAAZeEOCQAAAADAZSAkMFAtGje+YDn70SwVfPapfJqESObC+83IOSoNvE3Nm7esgFoCAFD9kEQHAAAAAKCaCQ286nTC3Y0kuiSVeLY6AABc1vhiUQAAAAAAAAAAXCCJDgAAAAAAAACAC0znAgAAAABADVZcUqJ0u71c2/BFpACAmoQ7HgAAAAAANVjmsRw5vtggn5BQt8rzRaQAgJqGJDpQRZWUFMtuzyzXNjwNAgAAAOBihAQGnv4iUjfxRaQAgJqEbBtQReUcytShokLZ6xW7VT77YIb6SzwNAgAAAMCjmP4FAFDTcAcDqrDAxqFq3KyFt6sBAAAAABamfwEA1DQk0QEAAAAAQLkw/QsAoCap5e0KAAAAAAAAAABQVfEkOgAAAAAAqDKKi4tlt/9crm2Ycx0A4EncYQAAAAAAQJVht/+sg6veVNPAhm6VZ851AICnkUQHqomSkmLZ7Znl2oanNQAAAABURU0DGzLnOgCgyiB7BlQTOYcydaioUPZ6xW6Vzz6Yof4ST2sAAAAA8KjikhKl2+1ul7fb7WomhwdrBABA+ZBEB6qRwMahatyshberAQAAAACWzGM5cnyxQT4hoW6Vz9q7W41Dm3q4VgAAuI8kOgAAAAAA8KiQwEC3p2exZ2d5uDYAAJQPSXSghirvHOrMnw4AAACgKirvdDES4xsAQPlwxwBqqPLMoc786QAAAACqqvJOF5ORc1QaeBvjGwCA20iiAzWYu3Oon35qnSc7AAAAAFRN5ZkuRpJKPFgXAED1Q4YLwAWV56l1STqSsV/t7XZdc801bh+DpDsAAACAysD0LwCA8qrWd4CCggI9+eST+vjjj+Xv769hw4Zp2LBh3q4WcFly96l1Sco6YNdX2afcTrozXQwAANUXMTmAqobpXwAA5VWtk+jTp0/X9u3btWzZMmVmZmrcuHEKCQlRz549vV01oNorT9K9vIqLi2W3/+y0rHZtHxUVuf5QZnmeHClr/xfCkykAAJSNmBxAVeSp6V8YSwBA9VRt36VPnTqlt956Sy+99JIiIyMVGRmp3bt3a/ny5QTsQBVzes71TLfL2+12fXuiREFNfmsts9UqlnGUXb6808uUtf/z4Ul6AADKRkwOoDpwNf1LWQ/y2O12+X69Wb9tGOTWvvdnZ8neoVOVmQqTPwK4RtsANVu1/U3euXOniouLFRsbay2Li4vTiy++KIfDoVq1anmxdgDOVt4513ftydBvW0U5Peleq5bkcJFEL+/0MmXtvyKVJ/gqLi6WZJOvr4/b+ydQAwBUFcTkAKqDMqd/sUmy2eRjjGT+uzhr725FhTZ1+yl3e3ZWlZpaxm7/WQdXvammgQ3dKl/ePwKUd3xjsxkVFzvk41Px46HyJsXL+wcSpgECqpdqm2XJysrSVVddpTp16ljLgoKCVFBQoNzcXAUGBnqxdgDOVd4516vS/iviSXpXdm37Wlc0CFRoM/cCr/I+dV/eIPZiyteu7StjLly2surjqfI17Q8ePImDMzx5LXCdXf6IyQFUF6Wmf7FJtWw2Oc5Jotuzsy593+dRXFIiezm+FLW8MardblezwKs89keA1L271TCgrlq4U94m/XPPHl0VEOBeeZUvqV/epHh5/0Di6b7y9PijvHFYVRprVbW29LSq1FdS1WqbilT9zug/8vLynIJ1SdbrwsJCt/djs1Votdw63tGDGZLD6T7s0rGsQyosKlSAf4Bb+69K5atSXdwpb5OkWrL6xtv1qczyVakuZZU/t28quz57//WtduTnq9HRfLfK2/+9XaFhrd0qW1652Yf1fkb56hJQv4EaNWnqkfLp/96ugCsbqFHjpm69p3m6Pp4sX959/3L0iG4Oc/8PHhXNZpN8fX1UXFzi9h85zma327V+l11XNrzarfLePt/LyaX2TWXz5LVwMfu+r5vUokXFP/F1Jias7NjwclcRMbk34vGMo0dPJ8bccOiXXBUVFiogwL244XIuX5H7tpKPlVT3mlb+UG6uioqK5O/n/bpU5/LeuI6//Xmf8nfukGnk3r1xe7pdDfz91bQc5R1NQmSzufdJoUO/5KphQN3/DMrKwUPlD/+Sq/y1q9xqn+3pdkU2CSlXXTJzcqpUX5Wn/JHjv8jeo49TDHa+uNNut+vw/63T1fWv9Hr9Pd02FdGWrlRGbF+V+urI8V+kP430SDzuSmXFijZjLofhWfl9+OGHeuqpp/Tll19ay/bu3atevXpp69atatCggfcqBwAAANQAxOQAAACoDqrtJITBwcE6duzYfz5ycFpWVpb8/f1Vv359L9YMAAAAqBmIyQEAAFAdVNskekREhHx9fbVt2zZrWWpqqqKiovgCIwAAAKASEJMDAACgOqi2kWtAQID69++vyZMn6/vvv9cnn3yiJUuW6O677/Z21QAAAIAagZgcAAAA1UG1nRNdOv1FRpMnT9bHH3+sevXq6b777tPQoUO9XS0AAACgxiAmBwAAwOWuWifRAQAAAAAAAAC4FNV2OhcAAAAAAAAAAC4VSXQAAAAAAAAAAFwgiQ4AAAAAAAAAgAsk0auQgoICPf7442rfvr0SEhK0ZMkSb1ep2issLFSfPn20detWa1l6erqGDh2qmJgY9erVS5s2bXLaZvPmzerTp4+io6N19913Kz093Wn90qVLlZiYqNjYWD3++OPKy8urlHOpLg4fPqxRo0YpPj5eiYmJmjZtmgoKCiTRN962f/9+3XfffYqNjdWNN96ol19+2VpH31QNw4cP1/jx463XO3bs0K233qro6GgNHDhQ27dvdyq/bt06devWTdHR0UpOTlZOTo61zhij5557Th07dlR8fLymT58uh8NRaedSXaxfv17h4eFOP6NGjZJE/3hTYWGhnnzySV133XW64YYbNGvWLJ35miD6pWYjHr90nnrfA2OXylBWGz/11FOlrunXX3/dWs99wT2M8zzvfG3MdVwxGBNXjvO1s1evZYMq429/+5vp27ev2b59u/n4449NbGys+fDDD71drWorPz/fJCcnm7CwMLNlyxZjjDEOh8P07dvXPPbYY2bPnj3mxRdfNNHR0ebAgQPGGGMOHDhgYmJizOLFi82uXbvMQw89ZPr06WMcDocxxpiPPvrIxMXFmU8//dSkpaWZXr16mSeffNJr53i5cTgc5g9/+IP54x//aHbt2mW++eYbc/PNN5tnnnmGvvGykpIS0717d/PYY4+Zn376yXz22WemXbt2Zu3atfRNFbFu3ToTFhZmxo0bZ4wx5tdffzWdOnUyzzzzjNmzZ4+ZMmWKueGGG8yvv/5qjDEmLS3NtG3b1rz77rvmxx9/NEOGDDHDhw+39rd48WKTlJRkvvnmG/PVV1+ZhIQE8/LLL3vl3C5nCxcuNCNGjDBHjhyxfn755Rf6x8smTpxounfvbtLS0szmzZtNhw4dzIoVK+gXEI9XAE+979V0jF08r6w2NsaYoUOHmpSUFKdr+tSpU8YY7gvuYpzneedrY2O4jisCY+LKcb52Nsa71zJJ9Cri119/NVFRUU436wULFpghQ4Z4sVbV1+7du02/fv1M3759nYKkzZs3m5iYGCuYN8aYe+65x8ydO9cYY8ycOXOc+uTUqVMmNjbW2v6OO+6wyhpjzDfffGPatm1r/ULj/Pbs2WPCwsJMVlaWtey9994zCQkJ9I2XHT582Dz00EPmxIkT1rLk5GQzadIk+qYKOHbsmOncubMZOHCglUR/6623TNeuXa3AzOFwmJtvvtmsWrXKGGPMmDFjrLLGGJOZmWnCw8ON3W43xhiTlJRklTXGmNWrV5suXbpU1ilVG4899piZOXNmqeX0j/ccO3bMtG7d2mzdutValpKSYsaPH0+/1HDE4xXDU+97NRljF89z1cbGGJOYmGi++OKLMrfjvuAexnmed742NobruCIwJq4c52tnY7x7LTOdSxWxc+dOFRcXKzY21loWFxentLS0GvsxGU/6+uuv1aFDB7355ptOy9PS0tS6dWvVrVvXWhYXF6dt27ZZ69u3b2+tCwgIUGRkpLZt26aSkhL961//clofExOjoqIi7dy507MnVE00atRIL7/8soKCgpyWnzx5kr7xsquvvlpz5sxRvXr1ZIxRamqqvvnmG8XHx9M3VcCzzz6rW265RS1btrSWpaWlKS4uTjabTZJks9nUrl07l/3SpEkThYSEKC0tTYcPH9bBgwd13XXXWevj4uJ04MABHTlypHJOqprYu3evmjVrVmo5/eM9qampqlevnuLj461lw4cP17Rp0+iXGo54vGJ44n2vpmPs4nmu2vjkyZM6fPhwmde0xH3BXYzzPO98bcx1XDEYE1eO87Wzt69lkuhVRFZWlq666irVqVPHWhYUFKSCggLl5uZ6r2LV1B133KHHH39cAQEBTsuzsrJ09dVXOy1r2LChDh06dMH1x48fV0FBgdN6X19fNWjQwNoe51e/fn0lJiZarx0Oh15//XV17NiRvqlCunbtqjvuuEOxsbHq0aMHfeNlX331lf75z39q5MiRTssv1C9HjhxxuT4rK0uSnNafCcjpF/cZY/TTTz9p06ZN6tGjh7p166bnnntOhYWF9I8XpaenKzQ0VKtXr1bPnj110003acGCBXI4HPRLDUc8fuk89b5X0zF28TxXbbx3717ZbDa9+OKL6ty5s/r166d3333XWs99wT2M8zzvfG3MdVzxGBNXjnPb2dvXsu+lnhAqRl5enlPALsl6XVhY6I0q1Uiu+uFMH5xvfX5+vvXa1fYonxkzZmjHjh16++23tXTpUvqmipg7d66ys7M1efJkTZs2jd8bLyooKNCkSZP0xBNPyN/f32ndhfolPz+/XP3CPan8MjMzrX6YM2eOMjIy9NRTTyk/P5/+8aJTp05p//79WrlypaZNm6asrCw98cQTCggIoF9qOOLxS+ep9z2UjRjM8/bt2yebzabmzZtryJAh+uabbzRx4kTVq1dPN998M/eFi8Q4z/PObuMffviB67iCMSauHOe2c2RkpFevZZLoVYSfn1+pTjvz+tzECDzHz8+v1JNGhYWFVh+46qf69evLz8/Pen3u+nOfaMCFzZgxQ8uWLdPs2bMVFhZG31QhUVFRkk4ncEePHq2BAweW+uZw+qZyzJ8/X23atHF66uQMV+1+oX4JCAhwCibO7SP6xX2hoaHaunWrrrzyStlsNkVERMjhcGjMmDGKj4+nf7zE19dXJ0+e1MyZMxUaGirpdOJvxYoVuvbaa+mXGox4/NJ56n0PZSM+9rz+/furS5cuatCggSSpVatW+vnnn7VixQrdfPPN3BcuAuM8zzu3jf/nf/6H67iCMSauHOe287fffuvVa5npXKqI4OBgHTt2TMXFxdayrKws+fv7q379+l6sWc0SHBys7Oxsp2XZ2dnWxz1crW/UqJEaNGggPz8/p/XFxcXKzc1Vo0aNPF/5amTKlCl65ZVXNGPGDPXo0UMSfeNt2dnZ+uSTT5yWtWzZUkVFRWrUqBF94yXvv/++PvnkE8XGxio2Nlbvvfee3nvvPcXGxl7S70xwcLAkWR95O/v/9Ev5NGjQwJr/V5JatGihgoKCS/q9oX8uTaNGjeTn52cl0CXpd7/7nQ4ePMjvTQ1HPF4xPPG+h7IRH3uezWazkjVnNG/eXIcPH5bEfaG8GOd5XlltzHVcMRgTV47ztfPJkye9ei2TRK8iIiIi5Ovra33pgHT6i6+ioqJUqxbdVFmio6P1ww8/WB/zkE73Q3R0tLU+NTXVWpeXl6cdO3YoOjpatWrVUlRUlNP6bdu2ydfXV61ataq8k7jMzZ8/XytXrtSsWbPUu3dvazl9410ZGRl64IEHrJuTJG3fvl2BgYGKi4ujb7zktdde03vvvafVq1dr9erV6tq1q7p27arVq1crOjpa3333nYwxkk7PU/vtt9+67JeDBw/q4MGDio6OVnBwsEJCQpzWp6amKiQkpNQcc3Dtiy++UIcOHZyeSvnxxx/VoEEDxcXF0T9eEh0drYKCAv3000/Wsn379ik0NJTfmxqOePzSeep9D2UjPva8559/XkOHDnVatnPnTjVv3lwS94XyYJznea7amOu4YjAmrhzna+fXXnvNu9eyQZUxceJE07t3b5OWlmbWr19v2rVrZ/7v//7P29Wq9sLCwsyWLVuMMcYUFxebXr16mYcfftjs2rXLpKSkmJiYGHPgwAFjjDHp6ekmKirKpKSkmF27dpmHHnrI9O3b1zgcDmOMMevWrTPt2rUz69evN2lpaaZ3795mypQpXju3y82ePXtMRESEmT17tjly5IjTD33jXcXFxWbAgAFm2LBhZvfu3eazzz4zN9xwg1m6dCl9U4WMGzfOjBs3zhhjzIkTJ0zHjh3NlClTzO7du82UKVNMp06dzK+//mqMMebbb781kZGR5u9//7v58ccfzZAhQ8yIESOsfaWkpJiEhASzZcsWs2XLFpOQkGCWLFnilfO6XJ04ccIkJiaaRx991Ozdu9d89tlnJiEhwSxatIj+8bLhw4eb2267zfz4449m48aNpmPHjmbZsmX0C4jHL5En3/dwGmMXzzu7jdPS0kzr1q3Nyy+/bPbv32+WL19u2rRpY7799ltjDPcFdzHO87zztTHXccVgTFw5ztfO3r6WSaJXIadOnTJjx441MTExJiEhwbzyyiverlKNcHaQZIwxP//8s7nzzjtNmzZtTO/evc2XX37pVP6zzz4z3bt3N23btjX33HOPsdvtTutTUlLM9ddfb+Li4syECRNMfn5+pZxHdZCSkmLCwsLK/DGGvvG2Q4cOmeTkZNOuXTvTqVMn88ILL1g3ffqmajg7iW7M6YFf//79TVRUlBk0aJD54YcfnMqvWrXKJCUlmZiYGJOcnGxycnKsdcXFxWbq1Kmmffv2pkOHDmbGjBlWf8N9u3btMkOHDjUxMTGmU6dOZt68eVY70j/ec/z4cTNmzBgTExNjrr/+evoFFuLxS+ep9z2cxtjF885t4/Xr15u+ffuaqKgo07Nnz1J/WOO+cGGM8zzvQm3MdVwxGBNXjvO1szevZZsx//k8HQAAAAAAAAAAcMLkfgAAAAAAAAAAuEASHQAAAAAAAAAAF0iiAwAAAAAAAADgAkl0AAAAAAAAAABcIIkOAAAAAAAAAIALJNEBAAAAAAAAAHCBJDoAAAAAAAAAAC6QRAcAVAhjjLercFmgnQAAAOApxJruoZ0AlBdJdABecddddyk8PNzpp1WrVmrXrp0GDBigNWvWeLuKVda8efMUHh7u7Wo42b17t26//XanZeHh4Zo3b95F7/PPf/6z3nrrLbfKvvPOOwoPD1dGRsZFH68ypKamavjw4ZVyrMLCQvXs2VPbtm2rlOMBAIDLDzH5xSMmL42YvDRicqD68PV2BQDUXK1bt9akSZOs1yUlJTp06JCWLl2qsWPHqkGDBkpKSvJiDeGujz76SN99912F7e+dd97R4cOHNXDgwArbZ1Xw1ltvae/evZVyrDp16mj06NEaN26c1qxZI39//0o5LgAAuLwQk1cfxOTuISYHcDF4Eh2A19SrV08xMTHWT1xcnHr37q0lS5aodu3aeuedd7xdRXhBfn6+nnvuOf35z39WrVrcpi5Ft27dVLt2ba1YscLbVQEAAFUUMTnKQkxecYjJgeqBd0IAVY6fn5/q1Kkjm81mLXM4HFq0aJFuvvlmtWnTRj169NBrr73mtJ3dbtef//xndejQQdHR0brtttv0+eefW+vnzZunrl27asOGDerZs6eio6P1hz/8QVu3bnXaz5EjRzRhwgQlJSWpbdu2GjRokP7xj384lQkPD9fy5cv1l7/8RfHx8YqNjdVDDz2k7Oxst+sjSbt27dKIESPUrl07tWvXTsnJyUpPT7/kNpSkzMxMPfroo4qPj1d0dLTuuece7dixw1qfkZGh8PBwffjhhxo1apRiY2MVHx+vv/71rzp16pRVrqioSM8995w6d+6stm3b6r777tPq1autj2rOmzdP8+fPt9rl7I+Lnjx50qmNRo0a5dRGZVm1apUKCgrUpUsXp+Wff/65Bg8erJiYGCUkJOiJJ57Q8ePHncqkpaVp8ODBioqK0o033qiXX37ZaX1GRobGjh2rhIQERUZG6vrrr9fYsWN17Ngxq0zXrl01depU3XPPPWrbtq3+8pe/SJJ27typBx54QB07dlRkZKQSExP11FNPKT8/39q2sLBQc+bM0U033aS2bduqT58+evfddyVJ48eP17vvvqsDBw4oPDzcGpAWFBRo+vTpSkpKUps2bdS3b1998MEHTvV2Vadly5apZ8+eioqKUmJioiZPnqyTJ086bdu3b1+98sorKiwsPG+7AwAAnI2YnJicmJyYHMB/kUQH4DXGGBUXF1s/BQUF2rdvnyZMmKBff/1Vt9xyi1V28uTJmjt3rvr166cXX3xRPXv21NSpU7VgwQJJpwP6ESNGKC8vT9OnT9fChQvVoEED3X///dq/f7+1n5ycHI0bN0533HGHnn/+efn7++u+++7Tjz/+KEnKzs7WoEGD9M9//lOPPPKI5s2bp9DQUCUnJ2vt2rVO9Z89e7YcDodmzZqlsWPHasOGDZo6darb9fnpp580ePBgHT16VM8++6yefvpppaen6/bbb9fRo0cvqW1zcnI0ePBg/fDDD5o4caJmzpwph8OhO++8s9RHFydNmqTQ0FAtXLhQ9913n95++2298MIL1vonnnhCy5Yt05AhQ7RgwQIFBQVp4sSJ1vpbb71VgwYNkiS9+eabuvXWW611r776qoqKivT888/rscce06effqq//e1v56372rVrdeONN6pOnTrWsg0bNmjEiBFq2LCh5syZo9GjR+uTTz7RI4884rTt5MmT1bt3by1atEixsbGaMWOGNmzYIEnKy8vT3Xffrb1792rSpElavHix7r77br3//vuaPXu2036WL1+uqKgoLVy4UIMGDdKRI0d05513Ki8vT88884xeeukl9e7dW6+99ppeffVVa7vRo0frlVde0a233qqUlBQlJCRo/PjxWrdunUaOHKmkpCQ1atRIb775pm688UYZY5ScnKyVK1fq3nvv1QsvvKDY2Fg98sgjWr169XnrtG7dOs2YMUN33nmnFi9erOTkZK1Zs0ZTpkxx2q5nz546fPiwvv766/O2OwAAqJmIyYnJy0JMTkwO4BwGALxgyJAhJiwsrNRPeHi46du3r/nwww+tsvv27TPh4eEmJSXFaR+zZ882UVFRJicnxxw5csSEhYWZtWvXWuuPHz9upk6danbt2mWMMWbu3LkmLCzMvPvuu1aZvLw806lTJ/Pwww8bY4yZPn26iYyMNBkZGU7Huueee0ynTp1MSUmJMcaYsLAwc/vttzuVGT9+vImJiTHGGLfq8+ijj5obbrjBnDhxwipz7NgxExcXZ5555hmXbXfmPM5n1qxZJioqyuk8CgoKzE033WQefPBBY4wx6enpJiwszIwePdpp27vuusv06dPHGGPM/v37TXh4uFmyZIlTmWHDhpmwsDCTnp7usk5hYWHm1ltvdVo2evRoc91117ms94kTJ0xERIR55ZVXnJb//ve/N/379zcOh8Na9v7775vu3bubrKwss2rVKhMWFmbeeOMNa/2pU6dMZGSkmTp1qjHGmB07dpjbb7/d2O12p32PGDHC9OjRw3rdpUsX061bN6cyX3zxhbnzzjud+soYY/r06WOGDRtmjDHm3//+twkLCzNLly51KvPAAw+Yv/71r8YYY8aNG2e6dOlirdu0aZMJCwsz77//fql26tSpkykqKnJZp4kTJ5oePXpY16QxxqxZs8a8+uqr5lzXXXedmT59eqnlAACgZiMmJyYvCzH5f9uJmBzAGXyxKACviYyM1JNPPinp9Mc158yZo6KiIs2ZM0fNmze3ym3ZskXGGHXt2lXFxcXW8q5du+qFF15QamqqbrrpJrVs2VITJ07Upk2blJCQoM6dO2vChAlOx/T19VWfPn2s1/7+/urcubM2btwoSfr6668VGxur0NBQp+369eunCRMmaN++fWrZsqUkKSYmxqlM48aNlZeXJ0kKCgq6YH22bNmi+Ph4+fv7W+dVr149tW/fXps3b76oNj3jq6++UkREhIKDg61916pVS507dy719E5Z53HgwAFJ0tatW2WMUc+ePZ3K9OnTR5s2bbpgPeLi4pxeN23atNTHPc928OBBlZSUqGnTptay/Px87dixQw8++KDTx4l79eqlXr16OW3fvn176/8BAQEKCgqyjhcREaE33nhDDodDP//8s/bv3689e/Zo3759TtfVmbJnS0hIUEJCgoqKirRnzx7t379fu3btUk5Ojho0aCBJSk1NlSR1797daduzP0p7rq+++ko2m01JSUmlru21a9dq9+7dVl3OrVPHjh315ptvasCAAerWrZuSkpLUt29fpzY6IyQkRBkZGS7rAQAAai5icmLycxGTn0ZMDuBsJNEBeM0VV1yhqKgo63V0dLT69eunYcOG6Z133lFgYKAkKTc3V5LUu3fvMvdz+PBh2Ww2LVmyRC+88ILWr1+v1atXq3bt2urWrZuefPJJXXnllZJOB9K+vs5vfQ0bNrSO8csvv+i3v/1tqWMEBQVJklOwGRAQ4FSmVq1aMsZIklv1yc3N1QcffFBqrj1J1rlfrNzcXO3fv1+RkZFlrj8zsLjQeeTk5Eg63UZnO/e1K3Xr1nW577KcOHGi1Ha//PKLjDFuHfN85yJJr7zyil588UXl5uYqKChIbdq0UUBAgHVcV/U+8xHh5cuX69SpU2rSpInatm0rPz8/q8yZa8jdtjmzjTFG7dq1K3P9kSNHrED93Dr16tVLDodDb7zxhhYuXGh9zHn06NGlBjIBAQGl5mUEAACQiMmJyUsjJndGTA5AIokOoAoJCgrSE088oYceekhPP/20Zs6cKUmqX7++pNNf2HLFFVeU2i4kJESSFBwcrMmTJ2vSpEnauXOnPvroI7300ku66qqrNGnSJEn/DarOlp2dbQVZV155pbKyskqVObPsqquucvt8LlSf3/zmN7rhhht07733ltr23EFFef3mN79RfHy8xo4dW+b6s+c2vNA5SKfb6Ew7S/8N5CvamfY9e2BUr1492Wy2UscsKCjQli1bFB0d7da+33vvPT3zzDMaM2aMBgwYYA2KHnroIf3rX/8677aLFi3S0qVL9eSTT6p79+76zW9+I0nWvJPSf6/TnJwcNW7c2Fq+d+9e5ebmlnoCSDrdT3Xr1nWaw/Fs11577Xnr1adPH/Xp00cnTpzQpk2b9NJLL2nMmDGKi4uz+k463Z5n9x8AAIArxOT/RUxOTC4RkwM4jS8WBVCl9OzZU4mJiVq3bp31pStnPg547NgxRUVFWT85OTl6/vnnlZubq++++0433HCDvv/+e9lsNkVEROiRRx5RWFiYMjMzrf3n5+friy++cHq9ceNGXX/99ZKk6667Tt9995310ckz1q5dq0aNGl0wgDrDnfrEx8drz549ioiIsM6pTZs2Wrp0qdavX3/xjfifff/000/63e9+59Rma9as0dtvvy0fHx+39hMXFycfH59S9fn444+dXteqVTG3k+DgYPn4+OjQoUPWsiuuuEIRERHWlxGdsXHjRg0fPlxHjhxxa9+pqamqX7++/vjHP1rB+q+//qrU1FQ5HI4LbtuyZUsNHDjQCtYPHz6sXbt2WdueCcg//fRTp22fe+45Pf3005JKt1N8fLxOnTolY4xTP+3atUsLFiwo9ZHWsz388MNKTk6WdDrw/9///V+NHDlSxcXFTm1ijNHhw4dLfRwaAADAFWJyYnJicmJyAM54Eh1AlfP444+rX79+euqpp/Tuu+8qPDxc/fr108SJE3XgwAG1adNGP/30k2bPnq2mTZuqWbNmKi4ulr+/v8aOHasHH3xQQUFB2rx5s3788UfdfffdTvufMGGCHn74YTVs2FCLFy/WqVOndP/990uS7r33Xq1du1ZDhw7VAw88oAYNGmj16tXasmWLpk6d6nZg2rp16wvWZ+TIkRo8eLBGjBih22+/XX5+fnrzzTf1ySefaO7cuRc8xtKlS0stq1+/vgYMGKChQ4dqzZo1Gjp0qIYNG6arrrpKH3zwgf7+97+XmpPyfH77299q4MCBmjVrloqKitSqVSutX7/eCp7PtMeZJz7WrVun6OjoMj9+6466deuqXbt2Sk1N1dChQ63lo0aN0v33369HH31U/fv3V3Z2tmbNmqVu3bopLCxM27dvv+C+27ZtqxUrVuiZZ55Rly5ddOTIES1evFjZ2dnWR4vPt+3ChQu1aNEixcTEaP/+/UpJSVFhYaH1MdxWrVqpZ8+emjFjhvLz8xUREaGNGzdqw4YNmj9/vtVO2dnZ+vzzzxUREaGkpCRdd911GjlypEaOHKkWLVro+++/19y5c5WYmHjejxB37NhRkyZN0rPPPqvOnTvr+PHjmj9/vpo1a6ZWrVpZ5Xbt2qUTJ04oMTHxgm0EAABwBjE5MTkxOTE5gP8iiQ6gymnevLnuuusuLVmyRCtWrNCQIUM0bdo0paSkaOXKlTp06JAaNmyoXr166eGHH5aPj498fHy0ZMkSzZw5U08//bSOHz+uZs2a6W9/+5sGDBjgtP/Jkydr6tSpysnJUbt27bRixQrraZZGjRppxYoVmjlzpp566ikrSF24cKFuuukmt8/Bz8/vgvVp1aqVli9frtmzZ2vs2LEyxigsLEwLFixw61jTpk0rteyaa67RgAEDFBwcrJUrV2rmzJmaPHmyCgoK1KxZMz399NNOH3d0x8SJE1W3bl0tWbJEJ0+e1PXXX6/7779fCxYssOYE7N69u9asWaPx48dr0KBBmjx5crmOcbYePXpo3rx5KigosOY37NKli1588UXNnz9fycnJCgwMVN++ffXggw+6vd/f//73ysjI0KpVq/TGG28oODhYSUlJuuOOOzRx4kTt3btXLVq0KHPbESNG6NixY3r11Ve1YMECNWnSRLfccotsNptSUlJ0/Phx1a9fXzNmzND8+fO1bNkyHTt2TC1atNDcuXPVrVs3SdKAAQP0+eefKzk5WaNGjdLw4cO1aNEiPf/880pJSdHRo0cVHByse++913qixZXBgwerqKhIK1eu1BtvvCF/f39df/31GjNmjGrXrm2V27hxoxo1auRyjkcAAICyEJMTkxOTE5MD+C+bOd+3SQBANTJv3jzNnz9f//73v71dlctGbm6uNm7cqMTERKe5J5999lm988472rp1a4UfMy8vT926ddOYMWPUv3//Ct9/TWKMUY8ePXTHHXc4PUUEAADgLcTk5UdMfnkjJgeqB55EBwC4FBAQoKeffloRERG65557VLduXW3btk2vv/66RowY4bFjPvjgg1q8eLH69u3r9lyRKO3jjz9WSUmJBg8e7O2qAAAA4CIRk1/eiMmB6oEvFgUAuOTn56elS5fKz89P48eP15/+9Ce99957Gjdu3AU/2ngpBg8erMaNG+utt97y2DGqu8LCQs2aNUvTp0+Xv7+/t6sDAACAi0RMfvkiJgeqD6ZzAQAAAAAAAADABZ5EBwAAAAAAAADABZLoAAAAAAAAAAC4QBIdAAAAAAAAAAAXSKIDAAAAAAAAAOACSXQAAAAAAAAAAFwgiQ4AAAAAAAAAgAsk0QEAAAAAAAAAcIEkOgAAAAAAAAAALpBEBwAAAAAAAADAhf8Hu2Tygjcw+SMAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 1500x600 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chosen responses - Mean length: 455.7, Median length: 343.0\n",
      "Rejected responses - Mean length: 349.0, Median length: 253.0\n",
      "Total chosen responses: 9999\n",
      "Total rejected responses: 9999\n"
     ]
    }
   ],
   "source": [
    "# 设置绘图风格\n",
    "plt.style.use('seaborn-v0_8')\n",
    "sns.set_palette(\"husl\")\n",
    "\n",
    "# 使用左右并列的两个子图\n",
    "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 6))\n",
    "\n",
    "# 绘制 chosen 长度分布\n",
    "chosen_lengths = list(chosen_len_cnt.keys())\n",
    "chosen_counts = list(chosen_len_cnt.values())\n",
    "\n",
    "ax1.hist(chosen_lengths, weights=chosen_counts, bins=50, alpha=0.7, color='skyblue', edgecolor='black')\n",
    "ax1.set_title('Distribution of Chosen Response Lengths', fontsize=14, fontweight='bold')\n",
    "ax1.set_xlabel('Response Length (characters)', fontsize=12)\n",
    "ax1.set_ylabel('Frequency', fontsize=12)\n",
    "ax1.grid(True, alpha=0.3)\n",
    "\n",
    "# 绘制 rejected 长度分布\n",
    "rejected_lengths = list(rejected_len_cnt.keys())\n",
    "rejected_counts = list(rejected_len_cnt.values())\n",
    "\n",
    "ax2.hist(rejected_lengths, weights=rejected_counts, bins=50, alpha=0.7, color='lightcoral', edgecolor='black')\n",
    "ax2.set_title('Distribution of Rejected Response Lengths', fontsize=14, fontweight='bold')\n",
    "ax2.set_xlabel('Response Length (characters)', fontsize=12)\n",
    "ax2.set_ylabel('Frequency', fontsize=12)\n",
    "ax2.grid(True, alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# 使用pandas Series直接计算统计信息\n",
    "chosen_series = pd.Series([length for length, count in chosen_len_cnt.items() for _ in range(count)])\n",
    "rejected_series = pd.Series([length for length, count in rejected_len_cnt.items() for _ in range(count)])\n",
    "\n",
    "# 展示统计信息\n",
    "print(f\"Chosen responses - Mean length: {chosen_series.mean():.1f}, Median length: {chosen_series.median():.1f}\")\n",
    "print(f\"Rejected responses - Mean length: {rejected_series.mean():.1f}, Median length: {rejected_series.median():.1f}\")\n",
    "print(f\"Total chosen responses: {len(chosen_series)}\")\n",
    "print(f\"Total rejected responses: {len(rejected_series)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40f3cbe1",
   "metadata": {},
   "source": [
    "### 3.3 评估模型基础能力\n",
    "定义评估标准。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eebe9a3b",
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate_dpo_model(testset_path):\n",
    "    output_lens = []\n",
    "    with open(testset_path, \"r\") as fin:\n",
    "        for line in fin:\n",
    "            data = json.loads(line.strip())\n",
    "            if data[-1].get(\"role\", \"\") == \"bot\":\n",
    "                output_lens.append(len(data[-1].get(\"content\", \"\")))\n",
    "    return sum(output_lens) / len(output_lens), sorted(output_lens)[len(output_lens) // 2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a3687fe",
   "metadata": {},
   "source": [
    "构造测试集。我们从原始数据中简单随机抽样200条，转为 OpenAI API 格式，作为测试集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "c3d68264",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ernie_data_to_openai_api_format(data, keep_system: bool = True):\n",
    "    openai_data = []\n",
    "    if keep_system and data.get(\"system\", \"\"):\n",
    "        openai_data.append({\"role\": \"system\", \"content\": data.get(\"system\", \"\")})\n",
    "    if len(data[\"src\"]) == 1:\n",
    "        openai_data.append({\"role\": \"user\", \"content\": data[\"src\"][0]})\n",
    "    else:\n",
    "        for i in range(len(data[\"src\"]) - 1):\n",
    "            openai_data.append({\"role\": \"user\", \"content\": data[\"src\"][i]})\n",
    "            openai_data.append({\"role\": \"assistant\", \"content\": data[\"tgt\"][i]})\n",
    "        openai_data.append({\"role\": \"user\", \"content\": data[\"src\"][-1]})\n",
    "    return openai_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "00ab689b",
   "metadata": {},
   "outputs": [],
   "source": [
    "testset_path = \"../data/dpo_testset.jsonl\"\n",
    "random.seed(RANDOM_SEED)\n",
    "\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    data_list = [json.loads(line) for line in fin]\n",
    "    test_data = random.sample(data_list, 200)\n",
    "    # 我们注意到原始数据中有些 system prompt 要求模型特意输出长文本（如『必须生成一个详细且长篇回答』），\n",
    "    # 这可能影响模型表现，因此我们不保留 system prompt\n",
    "    test_data = [ernie_data_to_openai_api_format(data, keep_system=False) for data in test_data]\n",
    "\n",
    "with open(testset_path, \"w\", encoding=\"utf-8\") as fout:\n",
    "    for item in test_data:\n",
    "        fout.write(json.dumps(item, ensure_ascii=False) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "262dbe2a",
   "metadata": {},
   "source": [
    "使用原始模型推理测试集。当前 WebUI 暂不支持推理测试集，我们使用 ERNIEKit 代码库中的推理脚本操作。\n",
    "- 脚本：`tools/inference/scripts/infer.sh`\n",
    "- 设置主要的超参数\n",
    "  - `top_p=0.7`\n",
    "  - `temperature=0.7`\n",
    "  - `max_seq_len=128000`（ERNIE-4.5-0.3B 支持动态128K 上下文长度）\n",
    "  - `max_dec_len=8192`（因为本任务关注模型本身的输出长度，所以要避免主动截断模型输出）\n",
    "  - `weight_quantize_algo` 如果训练时未使用量化可删去此行\n",
    "\n",
    "> 推理过程略"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6c3137d",
   "metadata": {},
   "source": [
    "计算评估指标。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5558bb29",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 获取推理结果\n",
    "dpo_infer_path = \"../data/infer/dpo_03b_exp0.jsonl\"\n",
    "raw_model_output_len_mean, raw_model_output_len_median = evaluate_dpo_model(dpo_infer_path)\n",
    "print(\"原始模型\")\n",
    "print(f\"输出长度均值：{raw_model_output_len_mean:.1f}，中位数：{raw_model_output_len_median}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a67ae5d",
   "metadata": {},
   "source": [
    "### 3.4 构造训练集\n",
    "怎样构造最有效呢？一般而言，训练集的 `rejected` 是待优化模型的真实输出，而 `chosen` 是基于此输出的优化结果。这里我们直接使用开源数据集的 `rejected`、`chosen`，不难想到有两种基本构造训练集思路，一种是 `rejected` 接近原始模型在测试集的输出长度且 `rejected` 显著长于 `chosen`；另一种不关心原始模型在测试集的输出长度，只要 `rejected` 显著长于 `chosen` 即可。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "effcd235",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练集1：rejected 长度接近原始模型\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    data_list = [json.loads(line) for line in fin]\n",
    "    trainset_1 = []\n",
    "    for data in data_list[201:]:\n",
    "        chosen = data[\"response\"][0]\n",
    "        rejected = data[\"response\"][1]\n",
    "        # 使用均值作为参考，允许一定范围的偏差\n",
    "        if (\n",
    "            abs(len(rejected) - raw_model_output_len_mean) < raw_model_output_len_mean * 0.4\n",
    "            and len(rejected) > len(chosen) * 1.2\n",
    "        ):\n",
    "            trainset_1.append(data)\n",
    "\n",
    "print(f\"训练集1样本数量：{len(trainset_1)}\")\n",
    "print(f\"训练集1中rejected平均长度：{sum(len(d['response'][1]) for d in trainset_1) / len(trainset_1):.1f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "cc5888a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 训练集2：rejected 显著长于 chosen\n",
    "with open(ernie_data_path, \"r\") as fin:\n",
    "    data_list = [json.loads(line) for line in fin]\n",
    "    trainset_2 = []\n",
    "    for data in data_list[201:]:\n",
    "        chosen = data[\"response\"][0]\n",
    "        rejected = data[\"response\"][1]\n",
    "        if len(rejected) > len(chosen) * 1.2:\n",
    "            trainset_2.append(data)\n",
    "\n",
    "print(f\"训练集2样本数量：{len(trainset_2)}\")\n",
    "print(f\"训练集2中rejected平均长度：{sum(len(d['response'][1]) for d in trainset_2) / len(trainset_2):.1f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64b74cba",
   "metadata": {},
   "source": [
    "控制变量，确保两个训练集的样本数量相同"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "cac5b121",
   "metadata": {},
   "outputs": [],
   "source": [
    "random.seed(RANDOM_SEED)\n",
    "trainset_2 = random.sample(trainset_2, len(trainset_1))\n",
    "\n",
    "print(f\"训练集2样本数量：{len(trainset_2)}\")\n",
    "print(f\"训练集2中rejected平均长度：{sum(len(d['response'][1]) for d in trainset_2) / len(trainset_2):.1f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e54f7d1",
   "metadata": {},
   "source": [
    "导出训练集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "4772b226",
   "metadata": {},
   "outputs": [],
   "source": [
    "trainset_1_path = \"../data/dpo_trainset_exp1.jsonl\"\n",
    "trainset_2_path = \"../data/dpo_trainset_exp2.jsonl\"\n",
    "\n",
    "# 我们注意到原始数据中有些 system prompt 要求模型特意输出长文本（如『必须生成一个详细且长篇回答』），\n",
    "# 这可能影响模型表现，因此我们不保留 system prompt\n",
    "with open(trainset_1_path, \"w\") as fout:\n",
    "    for data in trainset_1:\n",
    "        del data[\"system\"]\n",
    "        fout.write(json.dumps(data, ensure_ascii=False) + \"\\n\")\n",
    "\n",
    "with open(trainset_2_path, \"w\") as fout:\n",
    "    for data in trainset_2:\n",
    "        del data[\"system\"]\n",
    "        fout.write(json.dumps(data, ensure_ascii=False) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c0f1a02",
   "metadata": {},
   "source": [
    "# 4 训练\n",
    "### 4.1 选择基础模型\n",
    "考虑到展示的便利性与任务的复杂性，我们选择尺寸较小的 ERNIE-4.5-0.3B 模型。\n",
    "\n",
    "### 4.2 设计训练参数\n",
    "设计训练参数较为依赖经验。\n",
    "\n",
    "### 4.3 使用 WebUI 训练\n",
    "1. 在 ERNIEKit 项目的根目录启动 WebUI：`erniekit webui`  \n",
    "2. 配置模型路径及导出目录  \n",
    "\n",
    "![dpo_set_path](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_set_path_zh.png)\n",
    "\n",
    "3. 设置全参数/LoRA 精调、数值精度等  \n",
    "\n",
    "![dpo_set_lora](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_set_lora_zh.png)\n",
    "\n",
    "4. 设置精调模式（SFT/DPO），并配置训练参数  \n",
    "\n",
    "![dpo_train_params](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_train_params_zh.png)\n",
    "\n",
    "5. 配置训练集、验证集\n",
    "\n",
    "<div style=\"display: flex; justify-content: space-around;\">\n",
    "    <img src=\"https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_trainset_zh.png\" alt=\"dpo_trainset\" style=\"width: 49%;\">\n",
    "    <img src=\"https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_validset_zh.png\" alt=\"dpo_validset\" style=\"width: 49%;\">\n",
    "</div>\n",
    "\n",
    "### 4.4 查看训练日志及 loss 曲线\n",
    "训练日志路径：`${your_model_dir}/paddle_dist_log/workerlog.0`。也可以运行以下命令查看 loss 曲线：\n",
    "```bash\n",
    "visualdl --logdir ${your_model_dir}/vdl_log --host 0.0.0.0\n",
    "```\n",
    "\n",
    "### 4.5 合并模型权重（仅 LoRA 精调时需要）\n",
    "可以在 WebUI 的『评估』模式下方便地合并模型权重   \n",
    "\n",
    "![dpo_merge_lora](https://raw.githubusercontent.com/wiki/Minghao2812/ERNIE/img/dpo_merge_lora_zh.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62dcd43b",
   "metadata": {},
   "source": [
    "# 5 效果评估"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d53d017",
   "metadata": {},
   "source": [
    "### 5.1 计算评估指标\n",
    "上文已经定义了评估标准，并示范了计算评估指标之方法，此处不再复述，仅展示若干实验的训练配置以及评估指标，供读者参考。\n",
    "\n",
    "| 实验序号 | 训练集 | 训练参数 | ↓ 输出长度均值 | ↓ 输出长度中位数 |\n",
    "| --- | --- | --- | --- | --- |\n",
    "| 0 | - | - | 888.8 | 515 |\n",
    "| 1 | • len(rejected) 接近原始模型<br>• len(rejected) >= 1.2 len(chosen)<br>• 614Q | max_steps=614<br>warmup_steps=50<br>global_batch_size=1 | 879.5 | 442 |\n",
    "| 2 | • len(rejected) >= 1.2 len(chosen)<br>• 614Q | max_steps=614<br>warmup_steps=50<br>global_batch_size=1 | 864.7 | 354 |"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6639187",
   "metadata": {},
   "source": [
    "### 5.2 分析实验结果\n",
    "可以看到，同样的训练参数下，训练集2更为有效。于是自然想到：如果放入更多的训练数据，同时 `rejected` 与 `chosen` 的长度差异更显著，会不会令模型表现更佳？我们设计了第三组实验，实验结果如下：\n",
    "\n",
    "| 实验序号 | 训练集 | 训练参数 | ↓ 输出长度均值 | ↓ 输出长度中位数 |\n",
    "| ---- | ---- | ---- | ---- | ---- |\n",
    "| 0 | - | - | 888.8 | 515 |\n",
    "| 1 | len(rejected) 接近原始模型 <br> len(rejected) >= 1.2 len(chosen) <br> 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | 879.5 | 442 |\n",
    "| 2 | len(rejected) >= 1.2 len(chosen) <br> 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | 864.7 | 354 |\n",
    "| 3 | len(rejected) >= 2 len(chosen) <br> 1804Q | max_steps=1804 <br> warmup_steps=50 <br> global_batch_size=1 | 1145.1 | 412 |\n",
    "\n",
    "遗憾的是，模型表现变差了。有以下猜想：  \n",
    "1. warmup 不充分。根据一般经验 warmup 设为 max_steps 的10%可能较好。  \n",
    "2. max_steps 比较少。  \n",
    "\n",
    "针对以上两点猜想，我们设计了第四组实验。实验结果如下：\n",
    "\n",
    "| 实验序号 | 训练集 | 训练参数 | ↓ 输出长度均值 | ↓ 输出长度中位数 |\n",
    "| ---- | ---- | ---- | ---- | ---- |\n",
    "| 0 | - | - | 888.8 | 515 |\n",
    "| 1 | • len(rejected) 接近原始模型 <br> • len(rejected) >= 1.2 len(chosen) <br> • 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | 879.5 | 442 |\n",
    "| 2 | • len(rejected) >= 1.2 len(chosen) <br> • 614Q | max_steps=614 <br> warmup_steps=50 <br> global_batch_size=1 | 864.7 | 354 |\n",
    "| 3 | • len(rejected) >= 2 len(chosen) <br> • 1804Q | max_steps=1804 <br> warmup_steps=50 <br> global_batch_size=1 | 1145.1 | 412 |\n",
    "| 4 | • len(rejected) >= 2 len(chosen) <br> • 1804Q | max_steps=3608 <br> warmup_steps=360 <br> global_batch_size=1 | 628.1 | 331 |\n",
    "\n",
    "可以看到，第四组实验取得了更理想的效果！"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed3926fe",
   "metadata": {},
   "source": [
    "### 5.3 遗忘现象\n",
    "我们尝试测试模型在通用任务上的表现是否下降（出现『遗忘现象』）。好消息是，『遗忘』并不显著。例如：\n",
    "> 输入：[{\"role\": \"user\", \"content\": \"请记住，你的名字是 Sponge\"}, {\"role\": \"assistant\", \"content\": \"好的，我是 Sponge\"}, {\"role\": \"user\", \"content\": \"你是谁？\"}]  \n",
    "\n",
    "> ERNIE-4.5-0.3B 原始模型输出：我是 Sponge，一个超级好奇又有趣的家伙！有什么想和我聊的吗？  \n",
    "> DPO 模型输出：我是 Sponge，一个在海绵世界里探索的灵能战士！"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ar",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
